Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 925745 Details for
Bug 1128751
<driver/> isn't always formated as it should be
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
vdsm.log
vdsm.log (text/x-log), 10.35 MB, created by
Michael Burman
on 2014-08-11 13:16:04 UTC
(
hide
)
Description:
vdsm.log
Filename:
MIME Type:
Creator:
Michael Burman
Created:
2014-08-11 13:16:04 UTC
Size:
10.35 MB
patch
obsolete
>Dummy-33::DEBUG::2014-08-11 14:01:01,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:01,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133522 s, 76.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:01:02,551::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:01:02,552::task::595::Storage.TaskManager.Task::(_updateState) Task=`79f9b540-ccbc-495f-a918-d5e69d358f13`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:02,552::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:02,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:01:02,608::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:02,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`79f9b540-ccbc-495f-a918-d5e69d358f13`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:02,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`79f9b540-ccbc-495f-a918-d5e69d358f13`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:02,671::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:01:02,673::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:02,674::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:02,676::task::993::Storage.TaskManager.Task::(_decref) Task=`79f9b540-ccbc-495f-a918-d5e69d358f13`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:02,678::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:01:02,691::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:01:02,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`c275b3d2-99d5-4d96-947b-6d1edb8d2eaf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:01:02,695::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:01:02,698::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`14b784e7-07e1-476b-9cef-b8de037b4461`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:01:02,699::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:01:02,701::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:01:02,702::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`14b784e7-07e1-476b-9cef-b8de037b4461`::Granted request >Thread-34::DEBUG::2014-08-11 14:01:02,705::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c275b3d2-99d5-4d96-947b-6d1edb8d2eaf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:01:02,707::task::993::Storage.TaskManager.Task::(_decref) Task=`c275b3d2-99d5-4d96-947b-6d1edb8d2eaf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:01:02,786::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:02,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`c275b3d2-99d5-4d96-947b-6d1edb8d2eaf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:02,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`c275b3d2-99d5-4d96-947b-6d1edb8d2eaf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:01:02,796::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:01:02,799::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:01:02,805::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:01:02,810::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:01:02,813::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:01:02,816::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:01:02,823::task::993::Storage.TaskManager.Task::(_decref) Task=`c275b3d2-99d5-4d96-947b-6d1edb8d2eaf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:01:03,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:03,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226184 s, 45.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:05,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:05,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0235855 s, 43.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:01:06,880::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d71b36c-bee4-480e-a263-76ab768dc260`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:01:06,880::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:01:06,882::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:06,882::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:06,882::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:06,883::__init__::232::IOProcess::(_processLogs) (4192) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:01:06,887::__init__::232::IOProcess::(_processLogs) DEBUG|(4192) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:06,889::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:06,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:06,889::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:06,890::__init__::232::IOProcess::(_processLogs) (4193) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:01:06,890::__init__::232::IOProcess::(_processLogs) (4193) Queuing response >Thread-55::INFO::2014-08-11 14:01:06,890::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:01:06,890::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d71b36c-bee4-480e-a263-76ab768dc260`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:01:06,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d71b36c-bee4-480e-a263-76ab768dc260`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:01:06,891::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:01:06,891::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:01:06,891::task::993::Storage.TaskManager.Task::(_decref) Task=`8d71b36c-bee4-480e-a263-76ab768dc260`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:07,688::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:07,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:07,768::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:01:07,797::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:07,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:07,798::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:07,798::__init__::232::IOProcess::(_processLogs) (4194) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:07,799::__init__::232::IOProcess::(_processLogs) (4194) Queuing response >Thread-16::DEBUG::2014-08-11 14:01:07,799::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:01:07,822::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0039377 s, 116 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:07,823::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:07,824::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:07,824::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:07,824::__init__::232::IOProcess::(_processLogs) (4195) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:07,825::__init__::232::IOProcess::(_processLogs) DEBUG|(4195) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:07,826::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:07,826::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:07,827::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:07,827::__init__::232::IOProcess::(_processLogs) DEBUG|(4196) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:07,827::__init__::232::IOProcess::(_processLogs) (4196) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:07,828::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:07,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:07,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:07,829::__init__::232::IOProcess::(_processLogs) DEBUG|(4197) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:07,829::__init__::232::IOProcess::(_processLogs) (4197) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:07,839::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:07,839::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:07,839::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:07,840::__init__::232::IOProcess::(_processLogs) (4198) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:07,840::__init__::232::IOProcess::(_processLogs) DEBUG|(4198) Queuing response >Dummy-33::DEBUG::2014-08-11 14:01:07,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:07,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134215 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:09,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:09,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135969 s, 75.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:11,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:12,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131627 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:12,772::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:12,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:12,774::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:01:12,848::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:01:12,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ea36c12-c4fe-4bc0-aac7-a33cdd324d7b`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:12,867::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:01:12,880::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:12,880::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ea36c12-c4fe-4bc0-aac7-a33cdd324d7b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:12,881::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ea36c12-c4fe-4bc0-aac7-a33cdd324d7b`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:12,881::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:12,881::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:12,881::task::993::Storage.TaskManager.Task::(_decref) Task=`5ea36c12-c4fe-4bc0-aac7-a33cdd324d7b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:01:12,892::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:01:12,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd8078c0-cd70-4fe1-9e74-c38767b2ecf6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:01:12,893::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:01:12,894::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fbf50495-8c79-47da-ace0-328ea9ef5e92`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:01:12,894::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:01:12,894::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:01:12,894::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fbf50495-8c79-47da-ace0-328ea9ef5e92`::Granted request >Thread-34::DEBUG::2014-08-11 14:01:12,895::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bd8078c0-cd70-4fe1-9e74-c38767b2ecf6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:01:12,895::task::993::Storage.TaskManager.Task::(_decref) Task=`bd8078c0-cd70-4fe1-9e74-c38767b2ecf6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:01:12,915::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:12,915::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd8078c0-cd70-4fe1-9e74-c38767b2ecf6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:12,915::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd8078c0-cd70-4fe1-9e74-c38767b2ecf6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:01:12,916::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:01:12,916::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:01:12,916::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:01:12,917::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:01:12,917::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:01:12,917::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:01:12,917::task::993::Storage.TaskManager.Task::(_decref) Task=`bd8078c0-cd70-4fe1-9e74-c38767b2ecf6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:01:14,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:14,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131669 s, 77.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:01:15,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0dd1196-f610-4330-8e7f-59be71a5e0f3`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:15,602::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:01:15,602::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0039377', 'lastCheck': '7.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:01:15,602::task::1191::Storage.TaskManager.Task::(prepare) Task=`b0dd1196-f610-4330-8e7f-59be71a5e0f3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0039377', 'lastCheck': '7.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:01:15,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0dd1196-f610-4330-8e7f-59be71a5e0f3`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:15,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:15,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:15,603::task::993::Storage.TaskManager.Task::(_decref) Task=`b0dd1196-f610-4330-8e7f-59be71a5e0f3`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:01:15,622::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:01:16,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:16,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131851 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:17,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:17,781::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:17,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:01:17,842::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:17,842::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:17,842::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:17,843::__init__::232::IOProcess::(_processLogs) (4199) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:17,845::__init__::232::IOProcess::(_processLogs) (4199) Queuing response >Thread-16::DEBUG::2014-08-11 14:01:17,845::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:01:17,863::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418506 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:17,865::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:17,865::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:17,865::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:17,865::__init__::232::IOProcess::(_processLogs) (4200) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:17,867::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:17,867::__init__::232::IOProcess::(_processLogs) (4201) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:17,868::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:17,868::__init__::232::IOProcess::(_processLogs) (4202) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:17,869::__init__::232::IOProcess::(_processLogs) (4202) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:17,870::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:17,870::__init__::232::IOProcess::(_processLogs) (4203) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:01:18,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:18,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133814 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:20,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:20,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133346 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:22,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:22,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131377 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:22,788::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:22,789::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:22,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:01:22,933::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:01:22,933::task::595::Storage.TaskManager.Task::(_updateState) Task=`d198f4e3-959f-4f61-a76a-6092d5208c9e`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:22,934::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:01:22,954::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:22,955::task::1191::Storage.TaskManager.Task::(prepare) Task=`d198f4e3-959f-4f61-a76a-6092d5208c9e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:22,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`d198f4e3-959f-4f61-a76a-6092d5208c9e`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:22,955::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:22,955::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:22,956::task::993::Storage.TaskManager.Task::(_decref) Task=`d198f4e3-959f-4f61-a76a-6092d5208c9e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:01:22,966::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:01:22,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`a254d48d-dfac-4251-a4dc-6601fb77784d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:01:22,967::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:01:22,967::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c14b136b-8f1f-4644-8ab6-13ccd3f3d9c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:01:22,968::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:01:22,968::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:01:22,968::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c14b136b-8f1f-4644-8ab6-13ccd3f3d9c0`::Granted request >Thread-34::DEBUG::2014-08-11 14:01:22,969::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a254d48d-dfac-4251-a4dc-6601fb77784d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:01:22,969::task::993::Storage.TaskManager.Task::(_decref) Task=`a254d48d-dfac-4251-a4dc-6601fb77784d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:01:23,003::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:23,003::task::1191::Storage.TaskManager.Task::(prepare) Task=`a254d48d-dfac-4251-a4dc-6601fb77784d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:23,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`a254d48d-dfac-4251-a4dc-6601fb77784d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:01:23,004::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:01:23,004::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:01:23,004::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:01:23,005::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:01:23,005::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:01:23,005::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:01:23,006::task::993::Storage.TaskManager.Task::(_decref) Task=`a254d48d-dfac-4251-a4dc-6601fb77784d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:01:24,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:24,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131884 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:26,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:26,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133164 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:27,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:27,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:27,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:01:27,968::__init__::232::IOProcess::(_processLogs) (4204) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:01:27,972::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:27,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:27,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:01:27,993::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00276894 s, 164 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:27,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:27,995::__init__::232::IOProcess::(_processLogs) (4205) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:27,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:27,996::__init__::232::IOProcess::(_processLogs) (4206) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:27,997::__init__::232::IOProcess::(_processLogs) DEBUG|(4206) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:27,997::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:27,998::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:27,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:27,998::__init__::232::IOProcess::(_processLogs) DEBUG|(4207) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:27,999::__init__::232::IOProcess::(_processLogs) (4207) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:27,999::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:28,000::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:28,000::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:28,000::__init__::232::IOProcess::(_processLogs) (4208) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:28,000::__init__::232::IOProcess::(_processLogs) (4208) Queuing response >Dummy-33::DEBUG::2014-08-11 14:01:28,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:28,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132199 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:30,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:30,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134386 s, 76.2 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:01:30,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff76b55e-a74d-4f86-9bde-3ec56083e1a1`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:30,712::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:01:30,712::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00276894', 'lastCheck': '2.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:01:30,713::task::1191::Storage.TaskManager.Task::(prepare) Task=`ff76b55e-a74d-4f86-9bde-3ec56083e1a1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00276894', 'lastCheck': '2.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:01:30,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff76b55e-a74d-4f86-9bde-3ec56083e1a1`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:30,713::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:30,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:30,714::task::993::Storage.TaskManager.Task::(_decref) Task=`ff76b55e-a74d-4f86-9bde-3ec56083e1a1`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:01:30,732::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:01:32,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:32,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303203 s, 33.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:32,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:33,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:01:33,019::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:01:33,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`6885b531-d9ef-47d5-b7c3-a47240043959`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:33,072::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:33,072::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:01:33,087::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:33,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`6885b531-d9ef-47d5-b7c3-a47240043959`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:33,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`6885b531-d9ef-47d5-b7c3-a47240043959`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:33,087::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:33,088::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:33,088::task::993::Storage.TaskManager.Task::(_decref) Task=`6885b531-d9ef-47d5-b7c3-a47240043959`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:01:33,098::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:01:33,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`00a41681-2b75-4c0b-ba05-09c548c76c9d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:01:33,099::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:01:33,099::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`403d5451-18c9-41dc-95a1-004697710206`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:01:33,100::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:01:33,100::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:01:33,100::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`403d5451-18c9-41dc-95a1-004697710206`::Granted request >Thread-34::DEBUG::2014-08-11 14:01:33,101::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`00a41681-2b75-4c0b-ba05-09c548c76c9d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:01:33,101::task::993::Storage.TaskManager.Task::(_decref) Task=`00a41681-2b75-4c0b-ba05-09c548c76c9d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:01:33,122::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:33,123::task::1191::Storage.TaskManager.Task::(prepare) Task=`00a41681-2b75-4c0b-ba05-09c548c76c9d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:33,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`00a41681-2b75-4c0b-ba05-09c548c76c9d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:01:33,125::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:01:33,127::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:01:33,130::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:01:33,131::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:01:33,132::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:01:33,133::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:01:33,134::task::993::Storage.TaskManager.Task::(_decref) Task=`00a41681-2b75-4c0b-ba05-09c548c76c9d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:01:34,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:34,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132074 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:36,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:36,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132587 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:38,002::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:38,003::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:38,069::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:38,070::__init__::232::IOProcess::(_processLogs) (4209) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:01:38,070::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:01:38,072::__init__::232::IOProcess::(_processLogs) (4209) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:38,091::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:01:38,091::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0005 s, 910 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:38,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:38,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:01:38,097::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:38,098::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:38,098::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:38,098::__init__::232::IOProcess::(_processLogs) (4210) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:38,098::__init__::232::IOProcess::(_processLogs) DEBUG|(4210) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:38,100::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:38,100::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:38,100::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:38,100::__init__::232::IOProcess::(_processLogs) (4211) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:38,101::__init__::232::IOProcess::(_processLogs) DEBUG|(4211) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:38,101::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:38,102::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:38,102::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:38,102::__init__::232::IOProcess::(_processLogs) (4212) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:38,102::__init__::232::IOProcess::(_processLogs) DEBUG|(4212) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:38,103::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:38,104::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:38,104::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:38,104::__init__::232::IOProcess::(_processLogs) (4213) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:38,104::__init__::232::IOProcess::(_processLogs) DEBUG|(4213) Queuing response >Dummy-33::DEBUG::2014-08-11 14:01:38,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:38,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207918 s, 49.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:40,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:40,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134452 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:43,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:43,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207372 s, 49.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:43,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:43,101::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:43,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:01:43,151::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:01:43,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bd2ab3d-e0f7-4c50-9974-6077964a6452`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:43,153::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:01:43,165::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:43,166::task::1191::Storage.TaskManager.Task::(prepare) Task=`5bd2ab3d-e0f7-4c50-9974-6077964a6452`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:43,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bd2ab3d-e0f7-4c50-9974-6077964a6452`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:43,166::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:43,166::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:43,167::task::993::Storage.TaskManager.Task::(_decref) Task=`5bd2ab3d-e0f7-4c50-9974-6077964a6452`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:01:43,177::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:01:43,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`b338040c-00e5-4afc-9b25-bc27d378b770`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:01:43,178::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:01:43,179::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1e58142a-2ab4-477a-9db4-983cad7be42b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:01:43,179::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:01:43,180::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:01:43,180::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1e58142a-2ab4-477a-9db4-983cad7be42b`::Granted request >Thread-34::DEBUG::2014-08-11 14:01:43,180::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b338040c-00e5-4afc-9b25-bc27d378b770`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:01:43,181::task::993::Storage.TaskManager.Task::(_decref) Task=`b338040c-00e5-4afc-9b25-bc27d378b770`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:01:43,200::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:43,201::task::1191::Storage.TaskManager.Task::(prepare) Task=`b338040c-00e5-4afc-9b25-bc27d378b770`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079821312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:43,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`b338040c-00e5-4afc-9b25-bc27d378b770`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:01:43,201::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:01:43,201::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:01:43,202::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:01:43,202::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:01:43,202::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:01:43,202::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:01:43,203::task::993::Storage.TaskManager.Task::(_decref) Task=`b338040c-00e5-4afc-9b25-bc27d378b770`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:01:45,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:45,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131571 s, 77.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:01:45,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a871697-47d7-46e3-96a1-332347db0f31`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:45,816::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:01:45,817::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0005', 'lastCheck': '7.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:01:45,817::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a871697-47d7-46e3-96a1-332347db0f31`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0005', 'lastCheck': '7.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:01:45,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a871697-47d7-46e3-96a1-332347db0f31`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:45,818::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:45,818::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:45,818::task::993::Storage.TaskManager.Task::(_decref) Task=`2a871697-47d7-46e3-96a1-332347db0f31`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:01:45,837::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:01:47,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:47,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0229657 s, 44.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:48,106::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:48,107::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:48,107::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:48,107::__init__::232::IOProcess::(_processLogs) (4214) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:48,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:01:48,110::__init__::232::IOProcess::(_processLogs) DEBUG|(4214) Queuing response >Thread-16::DEBUG::2014-08-11 14:01:48,112::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:48,129::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:48,131::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:01:48,133::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00164232 s, 277 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:48,134::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:48,135::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:48,135::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:48,135::__init__::232::IOProcess::(_processLogs) (4215) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:48,136::__init__::232::IOProcess::(_processLogs) DEBUG|(4215) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:48,137::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:48,138::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:48,138::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:48,138::__init__::232::IOProcess::(_processLogs) (4216) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:48,139::__init__::232::IOProcess::(_processLogs) DEBUG|(4216) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:48,139::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:48,140::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:48,140::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:48,140::__init__::232::IOProcess::(_processLogs) (4217) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:48,140::__init__::232::IOProcess::(_processLogs) DEBUG|(4217) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:48,141::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:48,141::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:48,142::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:48,142::__init__::232::IOProcess::(_processLogs) (4218) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:48,143::__init__::232::IOProcess::(_processLogs) DEBUG|(4218) Queuing response >Dummy-33::DEBUG::2014-08-11 14:01:49,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:49,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131976 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:51,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:51,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192483 s, 53.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:53,137::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:01:53,214::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:01:53,271::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4c7839e-ad18-4093-a505-3aa1bc4a2d36`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:01:53,273::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:53,274::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:53,276::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:01:53,287::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:53,288::task::1191::Storage.TaskManager.Task::(prepare) Task=`c4c7839e-ad18-4093-a505-3aa1bc4a2d36`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:01:53,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4c7839e-ad18-4093-a505-3aa1bc4a2d36`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:01:53,288::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:01:53,288::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:01:53,288::task::993::Storage.TaskManager.Task::(_decref) Task=`c4c7839e-ad18-4093-a505-3aa1bc4a2d36`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:01:53,301::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:01:53,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`522b2ac4-0318-48d9-9d51-f3f6d93bc9d5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:01:53,305::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:01:53,309::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`33931d28-b5a6-41f5-87d6-1d3b48c17f21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:01:53,310::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:01:53,312::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:01:53,313::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`33931d28-b5a6-41f5-87d6-1d3b48c17f21`::Granted request >Thread-34::DEBUG::2014-08-11 14:01:53,316::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`522b2ac4-0318-48d9-9d51-f3f6d93bc9d5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:01:53,317::task::993::Storage.TaskManager.Task::(_decref) Task=`522b2ac4-0318-48d9-9d51-f3f6d93bc9d5`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:01:53,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 14:01:53,386::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:53,390::task::1191::Storage.TaskManager.Task::(prepare) Task=`522b2ac4-0318-48d9-9d51-f3f6d93bc9d5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:01:53,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`522b2ac4-0318-48d9-9d51-f3f6d93bc9d5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:01:53,394::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:01:53,395::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:01:53,402::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:01:53,407::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:01:53,410::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:01:53,414::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:01:53,417::task::993::Storage.TaskManager.Task::(_decref) Task=`522b2ac4-0318-48d9-9d51-f3f6d93bc9d5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:01:53,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0480195 s, 21.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:55,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:55,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214125 s, 47.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:01:57,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:57,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131632 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:58,144::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:58,144::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:58,145::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:58,145::__init__::232::IOProcess::(_processLogs) (4219) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:58,146::__init__::232::IOProcess::(_processLogs) DEBUG|(4219) Queuing response >Thread-16::DEBUG::2014-08-11 14:01:58,147::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:01:58,165::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000411197 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:01:58,166::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:01:58,166::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:01:58,167::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:58,167::__init__::232::IOProcess::(_processLogs) (4220) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:01:58,167::__init__::232::IOProcess::(_processLogs) (4220) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:58,169::__init__::232::IOProcess::(_processLogs) (4221) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:58,170::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:58,171::__init__::232::IOProcess::(_processLogs) (4222) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:01:58,171::__init__::232::IOProcess::(_processLogs) (4222) Queuing response >Thread-14::DEBUG::2014-08-11 14:01:58,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:01:58,172::__init__::232::IOProcess::(_processLogs) (4223) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:58,283::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:58,284::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:01:58,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:01:59,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:01:59,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131661 s, 77.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:02:00,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f928bb0-6521-4001-86ee-977f9156741d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:00,927::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:02:00,928::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411197', 'lastCheck': '2.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:00,928::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f928bb0-6521-4001-86ee-977f9156741d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411197', 'lastCheck': '2.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:00,928::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f928bb0-6521-4001-86ee-977f9156741d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:00,929::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:00,929::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:00,929::task::993::Storage.TaskManager.Task::(_decref) Task=`5f928bb0-6521-4001-86ee-977f9156741d`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:02:00,950::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:01,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:01,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133096 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:03,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:03,293::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:03,370::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:02:03,435::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:02:03,436::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc24ebd7-bfa9-4c4f-b0dd-2987f04630a1`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:03,436::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:02:03,449::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:03,450::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc24ebd7-bfa9-4c4f-b0dd-2987f04630a1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:03,450::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc24ebd7-bfa9-4c4f-b0dd-2987f04630a1`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:03,450::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:03,450::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:03,450::task::993::Storage.TaskManager.Task::(_decref) Task=`cc24ebd7-bfa9-4c4f-b0dd-2987f04630a1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:02:03,461::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:02:03,461::task::595::Storage.TaskManager.Task::(_updateState) Task=`f76866a1-6716-497a-9005-75dbe27fa93f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:02:03,462::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:02:03,462::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9e7f75c-7576-4793-9289-43ffdbfda401`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:02:03,463::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:02:03,463::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:02:03,463::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9e7f75c-7576-4793-9289-43ffdbfda401`::Granted request >Thread-34::DEBUG::2014-08-11 14:02:03,464::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f76866a1-6716-497a-9005-75dbe27fa93f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:02:03,464::task::993::Storage.TaskManager.Task::(_decref) Task=`f76866a1-6716-497a-9005-75dbe27fa93f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:02:03,485::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:03,485::task::1191::Storage.TaskManager.Task::(prepare) Task=`f76866a1-6716-497a-9005-75dbe27fa93f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:03,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`f76866a1-6716-497a-9005-75dbe27fa93f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:02:03,487::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:02:03,487::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:02:03,487::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:02:03,488::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:02:03,488::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:02:03,488::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:02:03,489::task::993::Storage.TaskManager.Task::(_decref) Task=`f76866a1-6716-497a-9005-75dbe27fa93f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:02:03,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:03,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131898 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:05,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:05,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131202 s, 78.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:02:07,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3fd5ad7-6230-4345-8b86-ccef7b3cf2b9`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:02:07,017::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:02:07,018::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:07,018::__init__::232::IOProcess::(_processLogs) (4224) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:02:07,022::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:07,022::__init__::232::IOProcess::(_processLogs) (4225) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:02:07,023::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:02:07,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`c3fd5ad7-6230-4345-8b86-ccef7b3cf2b9`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:02:07,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3fd5ad7-6230-4345-8b86-ccef7b3cf2b9`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:02:07,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:02:07,024::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:02:07,024::task::993::Storage.TaskManager.Task::(_decref) Task=`c3fd5ad7-6230-4345-8b86-ccef7b3cf2b9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:02:07,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:07,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225263 s, 45.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:08,174::__init__::232::IOProcess::(_processLogs) (4226) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:02:08,177::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:02:08,194::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423988 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:08,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:08,196::__init__::232::IOProcess::(_processLogs) (4227) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:08,197::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:08,198::__init__::232::IOProcess::(_processLogs) (4228) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:08,199::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:08,199::__init__::232::IOProcess::(_processLogs) (4229) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:08,200::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:08,200::__init__::232::IOProcess::(_processLogs) (4230) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:08,200::__init__::232::IOProcess::(_processLogs) (4230) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:08,374::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:08,376::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:08,377::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:10,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:10,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01311 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:12,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:12,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134308 s, 76.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:13,383::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:13,470::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:13,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:02:13,506::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:02:13,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdeffeff-88fe-490a-aa8f-d3c3bdb24b2f`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:13,507::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:02:13,520::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:13,520::task::1191::Storage.TaskManager.Task::(prepare) Task=`fdeffeff-88fe-490a-aa8f-d3c3bdb24b2f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:13,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdeffeff-88fe-490a-aa8f-d3c3bdb24b2f`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:13,520::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:13,521::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:13,521::task::993::Storage.TaskManager.Task::(_decref) Task=`fdeffeff-88fe-490a-aa8f-d3c3bdb24b2f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:02:13,530::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:02:13,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a6a0131-bc9b-4bdb-a4f5-1aa80a9802d7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:02:13,531::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:02:13,532::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3aae248e-69c4-423c-9f66-d4a74dcc8dcc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:02:13,532::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:02:13,533::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:02:13,533::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3aae248e-69c4-423c-9f66-d4a74dcc8dcc`::Granted request >Thread-34::DEBUG::2014-08-11 14:02:13,534::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6a6a0131-bc9b-4bdb-a4f5-1aa80a9802d7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:02:13,534::task::993::Storage.TaskManager.Task::(_decref) Task=`6a6a0131-bc9b-4bdb-a4f5-1aa80a9802d7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:02:13,554::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:13,554::task::1191::Storage.TaskManager.Task::(prepare) Task=`6a6a0131-bc9b-4bdb-a4f5-1aa80a9802d7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:13,554::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a6a0131-bc9b-4bdb-a4f5-1aa80a9802d7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:02:13,555::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:02:13,555::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:02:13,555::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:02:13,556::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:02:13,556::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:02:13,556::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:02:13,556::task::993::Storage.TaskManager.Task::(_decref) Task=`6a6a0131-bc9b-4bdb-a4f5-1aa80a9802d7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:02:14,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:14,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131426 s, 77.9 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:02:16,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`5fca005c-602b-4f8a-8b0f-baf032adf7ea`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:16,054::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:02:16,055::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423988', 'lastCheck': '7.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:16,055::task::1191::Storage.TaskManager.Task::(prepare) Task=`5fca005c-602b-4f8a-8b0f-baf032adf7ea`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423988', 'lastCheck': '7.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:16,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`5fca005c-602b-4f8a-8b0f-baf032adf7ea`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:16,055::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:16,055::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:16,056::task::993::Storage.TaskManager.Task::(_decref) Task=`5fca005c-602b-4f8a-8b0f-baf032adf7ea`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:02:16,074::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:16,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:16,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163468 s, 62.6 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:02:18,201::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:02:18,203::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,204::__init__::232::IOProcess::(_processLogs) (4231) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:02:18,212::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:02:18,213::__init__::232::IOProcess::(_processLogs) (4232) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 14:02:18,218::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:02:18,218::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,219::__init__::232::IOProcess::(_processLogs) (4233) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 14:02:18,222::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:02:18,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,223::__init__::232::IOProcess::(_processLogs) (4234) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:18,223::__init__::232::IOProcess::(_processLogs) DEBUG|(4234) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:18,224::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:18,225::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:18,225::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,225::__init__::232::IOProcess::(_processLogs) (4235) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:02:18,226::__init__::232::IOProcess::(_processLogs) DEBUG|(4235) Queuing response >Thread-16::DEBUG::2014-08-11 14:02:18,226::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 14:02:18,227::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:02:18,227::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:02:18,228::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:18,228::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:18,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,229::__init__::232::IOProcess::(_processLogs) (4236) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:18,229::__init__::232::IOProcess::(_processLogs) DEBUG|(4236) Queuing response >Thread-16::DEBUG::2014-08-11 14:02:18,230::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:18,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 14:02:18,265::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000644951 s, 705 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:18,266::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:18,267::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:18,267::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,267::__init__::232::IOProcess::(_processLogs) (4237) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:18,268::__init__::232::IOProcess::(_processLogs) (4237) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:18,269::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:18,270::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:18,271::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,272::__init__::232::IOProcess::(_processLogs) (4238) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:18,272::__init__::232::IOProcess::(_processLogs) (4238) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:18,272::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:18,273::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:18,274::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,275::__init__::232::IOProcess::(_processLogs) (4239) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:18,275::__init__::232::IOProcess::(_processLogs) (4239) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:18,275::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:18,276::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:18,276::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:18,277::__init__::232::IOProcess::(_processLogs) (4240) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:18,278::__init__::232::IOProcess::(_processLogs) (4240) Queuing response >Dummy-33::DEBUG::2014-08-11 14:02:18,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0149984 s, 68.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:18,477::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:18,478::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:18,479::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:20,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:20,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181441 s, 56.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:22,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:22,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131679 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:23,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:23,483::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:02:23,568::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:02:23,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`3caf2e85-dc96-4e3a-af49-3ec1cc6e6e89`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:23,573::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:23,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:02:23,588::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:23,588::task::1191::Storage.TaskManager.Task::(prepare) Task=`3caf2e85-dc96-4e3a-af49-3ec1cc6e6e89`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:23,588::task::595::Storage.TaskManager.Task::(_updateState) Task=`3caf2e85-dc96-4e3a-af49-3ec1cc6e6e89`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:23,589::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:23,589::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:23,589::task::993::Storage.TaskManager.Task::(_decref) Task=`3caf2e85-dc96-4e3a-af49-3ec1cc6e6e89`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:02:23,599::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:02:23,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`34f725b1-2032-4361-b8f9-5c92eeceaaf8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:02:23,600::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:02:23,601::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8d4a223a-d65f-485b-8a02-cce5dab141e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:02:23,601::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:02:23,601::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:02:23,602::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8d4a223a-d65f-485b-8a02-cce5dab141e5`::Granted request >Thread-34::DEBUG::2014-08-11 14:02:23,602::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`34f725b1-2032-4361-b8f9-5c92eeceaaf8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:02:23,602::task::993::Storage.TaskManager.Task::(_decref) Task=`34f725b1-2032-4361-b8f9-5c92eeceaaf8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:02:23,622::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:23,622::task::1191::Storage.TaskManager.Task::(prepare) Task=`34f725b1-2032-4361-b8f9-5c92eeceaaf8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:23,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`34f725b1-2032-4361-b8f9-5c92eeceaaf8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:02:23,623::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:02:23,623::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:02:23,623::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:02:23,624::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:02:23,624::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:02:23,624::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:02:23,625::task::993::Storage.TaskManager.Task::(_decref) Task=`34f725b1-2032-4361-b8f9-5c92eeceaaf8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:02:24,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:24,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204434 s, 50.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:26,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:26,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132784 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:28,279::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:28,280::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:02:28,282::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:02:28,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:28,300::__init__::232::IOProcess::(_processLogs) (4241) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:28,300::__init__::232::IOProcess::(_processLogs) (4241) Queuing response >Thread-16::DEBUG::2014-08-11 14:02:28,301::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000462722 s, 983 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:28,302::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:28,302::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:28,303::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:28,303::__init__::232::IOProcess::(_processLogs) (4242) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:28,304::__init__::232::IOProcess::(_processLogs) DEBUG|(4242) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:28,305::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:28,306::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:28,306::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:28,306::__init__::232::IOProcess::(_processLogs) (4243) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:28,307::__init__::232::IOProcess::(_processLogs) (4243) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:28,307::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:28,308::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:28,308::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:28,309::__init__::232::IOProcess::(_processLogs) (4244) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:28,309::__init__::232::IOProcess::(_processLogs) (4244) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:28,309::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:28,310::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:28,310::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:28,311::__init__::232::IOProcess::(_processLogs) (4245) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:28,311::__init__::232::IOProcess::(_processLogs) (4245) Queuing response >Dummy-33::DEBUG::2014-08-11 14:02:28,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:28,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131816 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:28,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:28,588::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:28,593::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:30,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:30,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013203 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:02:31,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e604361-addd-4065-b0e6-0019e7458811`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:31,207::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:02:31,207::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000462722', 'lastCheck': '2.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:31,207::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e604361-addd-4065-b0e6-0019e7458811`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000462722', 'lastCheck': '2.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:31,208::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e604361-addd-4065-b0e6-0019e7458811`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:31,208::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:31,208::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:31,208::task::993::Storage.TaskManager.Task::(_decref) Task=`8e604361-addd-4065-b0e6-0019e7458811`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:02:31,227::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:32,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:32,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0246724 s, 41.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:33,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:33,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:33,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:02:33,636::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:02:33,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`242d8f3d-b8a6-456c-80ec-a3b67c102213`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:33,638::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:02:33,651::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:33,651::task::1191::Storage.TaskManager.Task::(prepare) Task=`242d8f3d-b8a6-456c-80ec-a3b67c102213`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:33,652::task::595::Storage.TaskManager.Task::(_updateState) Task=`242d8f3d-b8a6-456c-80ec-a3b67c102213`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:33,652::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:33,652::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:33,652::task::993::Storage.TaskManager.Task::(_decref) Task=`242d8f3d-b8a6-456c-80ec-a3b67c102213`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:02:33,664::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:02:33,665::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c3a86a5-a3d8-4c54-ab6a-be75cdb86b43`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:02:33,665::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:02:33,666::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ec6760dd-616f-4e99-9192-805333a4f1af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:02:33,666::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:02:33,666::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:02:33,666::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ec6760dd-616f-4e99-9192-805333a4f1af`::Granted request >Thread-34::DEBUG::2014-08-11 14:02:33,667::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1c3a86a5-a3d8-4c54-ab6a-be75cdb86b43`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:02:33,667::task::993::Storage.TaskManager.Task::(_decref) Task=`1c3a86a5-a3d8-4c54-ab6a-be75cdb86b43`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:02:33,687::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:33,688::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c3a86a5-a3d8-4c54-ab6a-be75cdb86b43`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083425792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:33,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c3a86a5-a3d8-4c54-ab6a-be75cdb86b43`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:02:33,688::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:02:33,688::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:02:33,689::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:02:33,689::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:02:33,689::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:02:33,689::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:02:33,690::task::993::Storage.TaskManager.Task::(_decref) Task=`1c3a86a5-a3d8-4c54-ab6a-be75cdb86b43`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:02:34,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:34,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133376 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:36,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:36,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131556 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:38,312::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:38,312::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:38,313::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:38,313::__init__::232::IOProcess::(_processLogs) (4246) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:38,314::__init__::232::IOProcess::(_processLogs) DEBUG|(4246) Queuing response >Thread-16::DEBUG::2014-08-11 14:02:38,315::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:02:38,335::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00175889 s, 259 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:38,336::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:38,336::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:38,337::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:38,337::__init__::232::IOProcess::(_processLogs) (4247) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:38,338::__init__::232::IOProcess::(_processLogs) DEBUG|(4247) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:38,339::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:38,339::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:38,340::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:38,340::__init__::232::IOProcess::(_processLogs) (4248) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:38,340::__init__::232::IOProcess::(_processLogs) (4248) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:38,341::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:38,341::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:38,342::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:38,342::__init__::232::IOProcess::(_processLogs) (4249) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:38,342::__init__::232::IOProcess::(_processLogs) (4249) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:38,343::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:38,343::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:38,344::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:38,345::__init__::232::IOProcess::(_processLogs) (4250) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:38,345::__init__::232::IOProcess::(_processLogs) (4250) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:38,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:38,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:38,672::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:39,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:39,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204776 s, 50.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:41,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:41,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0214189 s, 47.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:43,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:43,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013263 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:43,679::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:43,680::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:02:43,704::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:02:43,774::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a4b0714-0927-428e-b2e2-361ac38fe784`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:43,775::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:43,776::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:02:43,790::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:43,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a4b0714-0927-428e-b2e2-361ac38fe784`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:43,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a4b0714-0927-428e-b2e2-361ac38fe784`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:43,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:43,791::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:43,791::task::993::Storage.TaskManager.Task::(_decref) Task=`7a4b0714-0927-428e-b2e2-361ac38fe784`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:02:43,803::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:02:43,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c2505f9-e438-4aa4-a0ff-33c2c314cf57`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:02:43,804::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:02:43,804::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ddbf7e58-d4f2-42fc-97b4-7b6179db35e8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:02:43,804::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:02:43,805::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:02:43,805::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ddbf7e58-d4f2-42fc-97b4-7b6179db35e8`::Granted request >Thread-34::DEBUG::2014-08-11 14:02:43,806::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0c2505f9-e438-4aa4-a0ff-33c2c314cf57`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:02:43,806::task::993::Storage.TaskManager.Task::(_decref) Task=`0c2505f9-e438-4aa4-a0ff-33c2c314cf57`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:02:43,826::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083360256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:43,826::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c2505f9-e438-4aa4-a0ff-33c2c314cf57`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863083360256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:43,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c2505f9-e438-4aa4-a0ff-33c2c314cf57`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:02:43,827::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:02:43,827::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:02:43,827::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:02:43,827::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:02:43,828::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:02:43,828::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:02:43,828::task::993::Storage.TaskManager.Task::(_decref) Task=`0c2505f9-e438-4aa4-a0ff-33c2c314cf57`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:02:45,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:45,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203724 s, 50.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:02:46,323::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f2e61cf-1772-4650-b91b-110d053a667b`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:46,324::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:02:46,324::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00175889', 'lastCheck': '8.0', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:46,325::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f2e61cf-1772-4650-b91b-110d053a667b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00175889', 'lastCheck': '8.0', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:02:46,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f2e61cf-1772-4650-b91b-110d053a667b`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:46,325::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:46,325::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:46,325::task::993::Storage.TaskManager.Task::(_decref) Task=`0f2e61cf-1772-4650-b91b-110d053a667b`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:02:46,344::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:47,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:47,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204878 s, 50.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:48,346::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:48,347::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:48,348::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:02:48,348::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:02:48,349::__init__::232::IOProcess::(_processLogs) (4251) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:48,366::__init__::232::IOProcess::(_processLogs) (4251) Queuing response >Thread-16::DEBUG::2014-08-11 14:02:48,368::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000472202 s, 964 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:48,369::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:48,369::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:48,369::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:48,369::__init__::232::IOProcess::(_processLogs) (4252) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:48,370::__init__::232::IOProcess::(_processLogs) DEBUG|(4252) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:48,372::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:48,372::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:48,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:48,373::__init__::232::IOProcess::(_processLogs) (4253) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:48,373::__init__::232::IOProcess::(_processLogs) (4253) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:48,374::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:48,374::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:48,375::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:48,375::__init__::232::IOProcess::(_processLogs) (4254) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:48,375::__init__::232::IOProcess::(_processLogs) (4254) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:48,376::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:48,376::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:48,377::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:48,378::__init__::232::IOProcess::(_processLogs) (4255) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:48,378::__init__::232::IOProcess::(_processLogs) (4255) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:48,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:48,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:48,872::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:49,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:49,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166145 s, 61.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:51,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:51,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192638 s, 53.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:53,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:53,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131675 s, 77.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:02:53,842::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:02:53,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d3efc12-7d10-4cd6-9f58-6b9a9ee6cb33`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:02:53,871::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:53,879::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:53,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:53,882::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:02:53,884::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:53,884::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d3efc12-7d10-4cd6-9f58-6b9a9ee6cb33`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:02:53,885::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d3efc12-7d10-4cd6-9f58-6b9a9ee6cb33`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:02:53,885::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:02:53,885::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:02:53,885::task::993::Storage.TaskManager.Task::(_decref) Task=`6d3efc12-7d10-4cd6-9f58-6b9a9ee6cb33`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:02:53,898::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:02:53,899::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d0b9963-eb77-47d8-9d5d-272032f5d0be`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:02:53,899::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:02:53,900::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e991117b-202c-4db4-abe4-370d443c0c32`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:02:53,900::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:02:53,900::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:02:53,901::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e991117b-202c-4db4-abe4-370d443c0c32`::Granted request >Thread-34::DEBUG::2014-08-11 14:02:53,901::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7d0b9963-eb77-47d8-9d5d-272032f5d0be`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:02:53,901::task::993::Storage.TaskManager.Task::(_decref) Task=`7d0b9963-eb77-47d8-9d5d-272032f5d0be`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:02:53,921::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:53,921::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d0b9963-eb77-47d8-9d5d-272032f5d0be`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:02:53,922::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d0b9963-eb77-47d8-9d5d-272032f5d0be`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:02:53,922::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:02:53,922::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:02:53,922::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:02:53,923::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:02:53,923::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:02:53,923::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:02:53,924::task::993::Storage.TaskManager.Task::(_decref) Task=`7d0b9963-eb77-47d8-9d5d-272032f5d0be`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:02:55,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:55,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131401 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:02:57,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:57,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131842 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:58,378::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:58,379::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:58,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:58,379::__init__::232::IOProcess::(_processLogs) (4256) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:58,380::__init__::232::IOProcess::(_processLogs) DEBUG|(4256) Queuing response >Thread-16::DEBUG::2014-08-11 14:02:58,381::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:02:58,400::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413983 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:02:58,402::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:58,402::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:58,402::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:58,403::__init__::232::IOProcess::(_processLogs) (4257) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:02:58,403::__init__::232::IOProcess::(_processLogs) DEBUG|(4257) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:58,405::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:58,405::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:58,405::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:58,405::__init__::232::IOProcess::(_processLogs) (4258) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:58,406::__init__::232::IOProcess::(_processLogs) DEBUG|(4258) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:58,407::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:58,407::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:58,408::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:58,408::__init__::232::IOProcess::(_processLogs) (4259) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:58,409::__init__::232::IOProcess::(_processLogs) (4259) Queuing response >Thread-14::DEBUG::2014-08-11 14:02:58,409::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:02:58,409::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:02:58,410::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:02:58,410::__init__::232::IOProcess::(_processLogs) (4260) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:02:58,411::__init__::232::IOProcess::(_processLogs) DEBUG|(4260) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:58,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:58,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:02:58,972::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:02:59,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:02:59,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204973 s, 50.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:01,434::task::595::Storage.TaskManager.Task::(_updateState) Task=`123b122e-03d4-4ccd-8d75-8091ef82d37c`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:01,434::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:03:01,437::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413983', 'lastCheck': '3.0', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:01,437::task::1191::Storage.TaskManager.Task::(prepare) Task=`123b122e-03d4-4ccd-8d75-8091ef82d37c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413983', 'lastCheck': '3.0', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:01,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`123b122e-03d4-4ccd-8d75-8091ef82d37c`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:01,437::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:01,438::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:01,438::task::993::Storage.TaskManager.Task::(_decref) Task=`123b122e-03d4-4ccd-8d75-8091ef82d37c`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:03:01,457::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:01,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:01,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135432 s, 75.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:03,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:03,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013238 s, 77.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:03,936::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:03:03,937::task::595::Storage.TaskManager.Task::(_updateState) Task=`218b6b95-dc97-47dd-97f9-ee498d8f5c85`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:03,972::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:03,979::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:03,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:03,982::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:03:03,996::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:03,997::task::1191::Storage.TaskManager.Task::(prepare) Task=`218b6b95-dc97-47dd-97f9-ee498d8f5c85`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:03,997::task::595::Storage.TaskManager.Task::(_updateState) Task=`218b6b95-dc97-47dd-97f9-ee498d8f5c85`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:03,997::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:03,997::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:03,998::task::993::Storage.TaskManager.Task::(_decref) Task=`218b6b95-dc97-47dd-97f9-ee498d8f5c85`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:03:04,010::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:03:04,011::task::595::Storage.TaskManager.Task::(_updateState) Task=`4866b336-be95-4d06-b9d8-b22253d8772e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:03:04,011::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:03:04,012::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8becf12d-36be-4b17-9806-b09034b658cd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:03:04,012::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:03:04,013::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:03:04,013::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8becf12d-36be-4b17-9806-b09034b658cd`::Granted request >Thread-34::DEBUG::2014-08-11 14:03:04,014::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4866b336-be95-4d06-b9d8-b22253d8772e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:03:04,014::task::993::Storage.TaskManager.Task::(_decref) Task=`4866b336-be95-4d06-b9d8-b22253d8772e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:03:04,046::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:04,046::task::1191::Storage.TaskManager.Task::(prepare) Task=`4866b336-be95-4d06-b9d8-b22253d8772e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:04,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`4866b336-be95-4d06-b9d8-b22253d8772e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:03:04,047::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:03:04,047::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:03:04,047::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:03:04,047::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:03:04,048::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:03:04,048::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:03:04,048::task::993::Storage.TaskManager.Task::(_decref) Task=`4866b336-be95-4d06-b9d8-b22253d8772e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:05,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:05,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205907 s, 49.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:03:07,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`120e95cd-ae51-444a-b5cf-f1503c2c58b4`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:03:07,128::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:03:07,129::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:07,129::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:07,130::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:07,130::__init__::232::IOProcess::(_processLogs) (4261) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:03:07,135::__init__::232::IOProcess::(_processLogs) DEBUG|(4261) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:07,136::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:07,137::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:03:07,137::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:03:07,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:03:07,138::task::1191::Storage.TaskManager.Task::(prepare) Task=`120e95cd-ae51-444a-b5cf-f1503c2c58b4`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:03:07,139::__init__::232::IOProcess::(_processLogs) (4262) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:03:07,140::task::595::Storage.TaskManager.Task::(_updateState) Task=`120e95cd-ae51-444a-b5cf-f1503c2c58b4`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:03:07,141::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:03:07,141::__init__::232::IOProcess::(_processLogs) (4262) Queuing response >Thread-55::DEBUG::2014-08-11 14:03:07,141::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:03:07,142::task::993::Storage.TaskManager.Task::(_decref) Task=`120e95cd-ae51-444a-b5cf-f1503c2c58b4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:07,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:07,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013233 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:08,412::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:08,413::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:08,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:08,413::__init__::232::IOProcess::(_processLogs) (4263) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:08,414::__init__::232::IOProcess::(_processLogs) DEBUG|(4263) Queuing response >Thread-16::DEBUG::2014-08-11 14:03:08,415::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:03:08,437::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00172499 s, 264 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:08,443::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:08,445::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:08,445::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:08,445::__init__::232::IOProcess::(_processLogs) (4264) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:08,446::__init__::232::IOProcess::(_processLogs) DEBUG|(4264) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:08,447::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:08,448::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:08,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:08,449::__init__::232::IOProcess::(_processLogs) DEBUG|(4265) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:08,449::__init__::232::IOProcess::(_processLogs) (4265) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:08,450::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:08,450::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:08,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:08,450::__init__::232::IOProcess::(_processLogs) (4266) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:08,451::__init__::232::IOProcess::(_processLogs) DEBUG|(4266) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:08,451::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:08,452::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:08,452::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:08,453::__init__::232::IOProcess::(_processLogs) (4267) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:08,454::__init__::232::IOProcess::(_processLogs) (4267) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:08,985::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:08,986::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:09,073::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:09,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:09,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133238 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:11,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:11,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206271 s, 49.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:13,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:14,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131155 s, 78.1 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:14,062::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:03:14,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`03eb3b91-baff-4190-a355-40a4bf53f0fd`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:14,076::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:14,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:03:14,091::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:14,093::task::1191::Storage.TaskManager.Task::(prepare) Task=`03eb3b91-baff-4190-a355-40a4bf53f0fd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:14,093::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:03:14,095::task::595::Storage.TaskManager.Task::(_updateState) Task=`03eb3b91-baff-4190-a355-40a4bf53f0fd`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:14,096::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:14,097::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:14,099::task::993::Storage.TaskManager.Task::(_decref) Task=`03eb3b91-baff-4190-a355-40a4bf53f0fd`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:14,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:03:14,114::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:03:14,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c95e57a-e08a-413e-a653-bb7e15a175d0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:03:14,117::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:03:14,120::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`93598504-39d7-4433-b279-a44ad7f74257`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:03:14,121::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:03:14,123::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:03:14,124::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`93598504-39d7-4433-b279-a44ad7f74257`::Granted request >Thread-34::DEBUG::2014-08-11 14:03:14,126::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5c95e57a-e08a-413e-a653-bb7e15a175d0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:03:14,127::task::993::Storage.TaskManager.Task::(_decref) Task=`5c95e57a-e08a-413e-a653-bb7e15a175d0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:03:14,184::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:14,185::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c95e57a-e08a-413e-a653-bb7e15a175d0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:14,186::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c95e57a-e08a-413e-a653-bb7e15a175d0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:03:14,188::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:03:14,189::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:03:14,192::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:03:14,193::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:03:14,194::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:03:14,195::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:03:14,198::task::993::Storage.TaskManager.Task::(_decref) Task=`5c95e57a-e08a-413e-a653-bb7e15a175d0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:16,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:16,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131937 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:16,552::task::595::Storage.TaskManager.Task::(_updateState) Task=`761119cb-00ed-4413-82a9-e60f3f69f2e2`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:16,553::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:03:16,553::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00172499', 'lastCheck': '8.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:16,553::task::1191::Storage.TaskManager.Task::(prepare) Task=`761119cb-00ed-4413-82a9-e60f3f69f2e2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00172499', 'lastCheck': '8.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:16,554::task::595::Storage.TaskManager.Task::(_updateState) Task=`761119cb-00ed-4413-82a9-e60f3f69f2e2`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:16,554::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:16,554::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:16,554::task::993::Storage.TaskManager.Task::(_decref) Task=`761119cb-00ed-4413-82a9-e60f3f69f2e2`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:03:16,573::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:18,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:18,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131131 s, 78.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:18,454::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:18,455::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:18,455::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:18,455::__init__::232::IOProcess::(_processLogs) (4268) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:18,455::__init__::232::IOProcess::(_processLogs) (4268) Queuing response >Thread-16::DEBUG::2014-08-11 14:03:18,456::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:03:18,494::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000391814 s, 1.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:18,498::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:18,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:18,499::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:18,499::__init__::232::IOProcess::(_processLogs) (4269) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:18,512::__init__::232::IOProcess::(_processLogs) DEBUG|(4269) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:18,516::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:18,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:18,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:18,517::__init__::232::IOProcess::(_processLogs) (4270) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:18,517::__init__::232::IOProcess::(_processLogs) (4270) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:18,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:18,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:18,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:18,519::__init__::232::IOProcess::(_processLogs) (4271) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:18,519::__init__::232::IOProcess::(_processLogs) (4271) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:18,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:18,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:18,521::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:18,521::__init__::232::IOProcess::(_processLogs) (4272) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:18,521::__init__::232::IOProcess::(_processLogs) (4272) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:19,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:19,112::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:19,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:20,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:20,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131777 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:22,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:22,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131006 s, 78.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:24,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:24,179::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:03:24,214::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:03:24,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`42580e62-fbe7-44dd-aa38-02ab8235f912`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:24,277::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:03:24,278::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:03:24,295::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:24,297::task::1191::Storage.TaskManager.Task::(prepare) Task=`42580e62-fbe7-44dd-aa38-02ab8235f912`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:24,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`42580e62-fbe7-44dd-aa38-02ab8235f912`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:24,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:24,301::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:24,303::task::993::Storage.TaskManager.Task::(_decref) Task=`42580e62-fbe7-44dd-aa38-02ab8235f912`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:03:24,319::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:03:24,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`1076e3c8-bfab-4de7-8b27-a832e90721cc`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:03:24,323::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:03:24,327::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f58b26b7-a619-43f0-8c75-7ae8acd6e542`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:03:24,328::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:03:24,330::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:03:24,332::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f58b26b7-a619-43f0-8c75-7ae8acd6e542`::Granted request >Thread-34::DEBUG::2014-08-11 14:03:24,334::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1076e3c8-bfab-4de7-8b27-a832e90721cc`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:03:24,336::task::993::Storage.TaskManager.Task::(_decref) Task=`1076e3c8-bfab-4de7-8b27-a832e90721cc`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:24,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 14:03:24,409::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:24,410::task::1191::Storage.TaskManager.Task::(prepare) Task=`1076e3c8-bfab-4de7-8b27-a832e90721cc`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:24,410::task::595::Storage.TaskManager.Task::(_updateState) Task=`1076e3c8-bfab-4de7-8b27-a832e90721cc`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:03:24,411::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:03:24,411::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:03:24,412::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:03:24,413::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:03:24,414::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:03:24,417::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:03:24,420::task::993::Storage.TaskManager.Task::(_decref) Task=`1076e3c8-bfab-4de7-8b27-a832e90721cc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:24,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0322327 s, 31.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:26,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:26,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138307 s, 74.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:28,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:03:28,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:03:28,525::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:03:28,526::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:03:28,526::__init__::232::IOProcess::(_processLogs) (4273) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:28,543::__init__::232::IOProcess::(_processLogs) (4273) Queuing response >Thread-16::DEBUG::2014-08-11 14:03:28,545::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417685 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:28,546::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:28,547::__init__::232::IOProcess::(_processLogs) (4274) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:28,548::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:28,548::__init__::232::IOProcess::(_processLogs) (4275) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:28,549::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:28,549::__init__::232::IOProcess::(_processLogs) (4276) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:28,550::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:28,551::__init__::232::IOProcess::(_processLogs) (4277) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:03:28,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:28,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132665 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:29,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:29,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:29,373::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:30,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:30,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134076 s, 76.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:31,666::task::595::Storage.TaskManager.Task::(_updateState) Task=`882dc9e1-66eb-4e13-9daf-967810976c9d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:31,667::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:03:31,667::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417685', 'lastCheck': '3.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:31,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`882dc9e1-66eb-4e13-9daf-967810976c9d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417685', 'lastCheck': '3.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:31,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`882dc9e1-66eb-4e13-9daf-967810976c9d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:31,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:31,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:31,668::task::993::Storage.TaskManager.Task::(_decref) Task=`882dc9e1-66eb-4e13-9daf-967810976c9d`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:03:31,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:32,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:32,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274234 s, 37.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:34,377::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:34,378::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:03:34,438::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-81::DEBUG::2014-08-11 14:03:34,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b32c481-89a3-49b8-b6a3-fc269eb059ba`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:34,477::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:34,477::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:03:34,492::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:34,492::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b32c481-89a3-49b8-b6a3-fc269eb059ba`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:34,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b32c481-89a3-49b8-b6a3-fc269eb059ba`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:34,493::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:34,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:34,493::task::993::Storage.TaskManager.Task::(_decref) Task=`1b32c481-89a3-49b8-b6a3-fc269eb059ba`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:03:34,503::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-34::DEBUG::2014-08-11 14:03:34,504::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e9ac277-cdfe-478f-a457-e4ae50cd3cdc`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:03:34,504::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:03:34,505::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0b8d63ee-af67-4fc4-8c50-9646548f0207`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:03:34,505::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:03:34,505::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:03:34,506::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0b8d63ee-af67-4fc4-8c50-9646548f0207`::Granted request >Thread-34::DEBUG::2014-08-11 14:03:34,506::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4e9ac277-cdfe-478f-a457-e4ae50cd3cdc`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:03:34,506::task::993::Storage.TaskManager.Task::(_decref) Task=`4e9ac277-cdfe-478f-a457-e4ae50cd3cdc`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:03:34,526::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:34,527::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e9ac277-cdfe-478f-a457-e4ae50cd3cdc`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:34,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e9ac277-cdfe-478f-a457-e4ae50cd3cdc`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:03:34,527::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:03:34,527::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:03:34,528::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:03:34,528::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:03:34,528::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:03:34,528::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:03:34,529::task::993::Storage.TaskManager.Task::(_decref) Task=`4e9ac277-cdfe-478f-a457-e4ae50cd3cdc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:34,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:34,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132867 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:36,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:37,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137444 s, 74.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:38,552::__init__::232::IOProcess::(_processLogs) (4278) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:03:38,555::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:03:38,573::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00174303 s, 261 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:38,575::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:38,575::__init__::232::IOProcess::(_processLogs) (4279) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:38,577::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:38,578::__init__::232::IOProcess::(_processLogs) (4280) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:38,578::__init__::232::IOProcess::(_processLogs) (4280) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:38,579::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:38,579::__init__::232::IOProcess::(_processLogs) (4281) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:38,580::__init__::232::IOProcess::(_processLogs) (4281) Queuing response >Thread-14::DEBUG::2014-08-11 14:03:38,580::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:38,581::__init__::232::IOProcess::(_processLogs) (4282) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:03:39,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:39,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131882 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:39,485::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:39,486::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:39,573::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:41,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:41,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205548 s, 49.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:43,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:43,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134144 s, 76.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:44,540::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:03:44,541::task::595::Storage.TaskManager.Task::(_updateState) Task=`20a602b1-e8c7-44b4-a684-f0ef9e634830`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:44,572::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:44,580::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:44,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:44,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:03:44,585::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:44,586::task::1191::Storage.TaskManager.Task::(prepare) Task=`20a602b1-e8c7-44b4-a684-f0ef9e634830`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:44,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`20a602b1-e8c7-44b4-a684-f0ef9e634830`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:44,586::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:44,586::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:44,586::task::993::Storage.TaskManager.Task::(_decref) Task=`20a602b1-e8c7-44b4-a684-f0ef9e634830`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:03:44,596::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:03:44,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`3646a39b-d293-4fae-a50e-23ca579528d6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:03:44,597::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:03:44,598::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`089e999b-3cd6-43f3-9e4b-1f6a3ebff96d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:03:44,598::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:03:44,599::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:03:44,599::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`089e999b-3cd6-43f3-9e4b-1f6a3ebff96d`::Granted request >Thread-34::DEBUG::2014-08-11 14:03:44,599::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3646a39b-d293-4fae-a50e-23ca579528d6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:03:44,600::task::993::Storage.TaskManager.Task::(_decref) Task=`3646a39b-d293-4fae-a50e-23ca579528d6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:03:44,619::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:44,620::task::1191::Storage.TaskManager.Task::(prepare) Task=`3646a39b-d293-4fae-a50e-23ca579528d6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086571520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:44,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`3646a39b-d293-4fae-a50e-23ca579528d6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:03:44,620::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:03:44,620::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:03:44,621::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:03:44,621::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:03:44,621::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:03:44,621::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:03:44,622::task::993::Storage.TaskManager.Task::(_decref) Task=`3646a39b-d293-4fae-a50e-23ca579528d6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:45,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:45,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203616 s, 50.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:46,784::task::595::Storage.TaskManager.Task::(_updateState) Task=`de1288ea-a00b-4aa0-b5a1-cde83cef0ce2`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:46,784::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:03:46,785::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00174303', 'lastCheck': '8.2', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:46,785::task::1191::Storage.TaskManager.Task::(prepare) Task=`de1288ea-a00b-4aa0-b5a1-cde83cef0ce2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00174303', 'lastCheck': '8.2', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:03:46,785::task::595::Storage.TaskManager.Task::(_updateState) Task=`de1288ea-a00b-4aa0-b5a1-cde83cef0ce2`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:46,785::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:46,786::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:46,786::task::993::Storage.TaskManager.Task::(_decref) Task=`de1288ea-a00b-4aa0-b5a1-cde83cef0ce2`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:03:46,811::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:47,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:47,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181985 s, 56.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:48,583::__init__::232::IOProcess::(_processLogs) (4283) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:03:48,584::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:03:48,601::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000442869 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:48,602::__init__::232::IOProcess::(_processLogs) (4284) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:48,604::__init__::232::IOProcess::(_processLogs) (4285) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:48,605::__init__::232::IOProcess::(_processLogs) (4286) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:48,606::__init__::232::IOProcess::(_processLogs) (4287) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:03:49,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:49,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132488 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:49,588::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:49,589::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:49,673::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:03:51,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:51,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133479 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:53,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:53,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0264462 s, 38.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:03:54,634::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:03:54,635::task::595::Storage.TaskManager.Task::(_updateState) Task=`ddfb1d27-6964-46d3-aedb-fc1ce1af8637`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:03:54,672::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:54,680::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:54,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:54,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:03:54,685::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:54,686::task::1191::Storage.TaskManager.Task::(prepare) Task=`ddfb1d27-6964-46d3-aedb-fc1ce1af8637`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:03:54,686::task::595::Storage.TaskManager.Task::(_updateState) Task=`ddfb1d27-6964-46d3-aedb-fc1ce1af8637`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:03:54,686::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:03:54,686::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:03:54,686::task::993::Storage.TaskManager.Task::(_decref) Task=`ddfb1d27-6964-46d3-aedb-fc1ce1af8637`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:03:54,697::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:03:54,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfdaa76e-0167-45a8-a99e-42f2b44485e5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:03:54,698::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:03:54,699::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`185fec95-6f02-4ef6-8d6a-23f82b0e18b7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:03:54,699::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:03:54,699::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:03:54,699::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`185fec95-6f02-4ef6-8d6a-23f82b0e18b7`::Granted request >Thread-34::DEBUG::2014-08-11 14:03:54,700::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bfdaa76e-0167-45a8-a99e-42f2b44485e5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:03:54,700::task::993::Storage.TaskManager.Task::(_decref) Task=`bfdaa76e-0167-45a8-a99e-42f2b44485e5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:03:54,722::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:54,722::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfdaa76e-0167-45a8-a99e-42f2b44485e5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:03:54,723::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfdaa76e-0167-45a8-a99e-42f2b44485e5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:03:54,723::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:03:54,723::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:03:54,723::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:03:54,724::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:03:54,724::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:03:54,724::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:03:54,724::task::993::Storage.TaskManager.Task::(_decref) Task=`bfdaa76e-0167-45a8-a99e-42f2b44485e5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:03:55,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:55,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205262 s, 49.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:03:57,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:57,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220598 s, 46.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:58,608::__init__::232::IOProcess::(_processLogs) (4288) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:03:58,609::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:03:58,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000459035 s, 991 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:03:58,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:58,629::__init__::232::IOProcess::(_processLogs) (4289) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:03:58,630::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:58,631::__init__::232::IOProcess::(_processLogs) (4290) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:58,632::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:58,632::__init__::232::IOProcess::(_processLogs) (4291) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:03:58,633::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:03:58,633::__init__::232::IOProcess::(_processLogs) (4292) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:03:59,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:03:59,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132292 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:59,690::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:59,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:03:59,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:01,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:01,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0204994 s, 50.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:01,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`a073fa5e-aad4-4f76-bbc1-2bc74fa0dc33`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:01,908::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:04:01,908::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000459035', 'lastCheck': '3.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:01,909::task::1191::Storage.TaskManager.Task::(prepare) Task=`a073fa5e-aad4-4f76-bbc1-2bc74fa0dc33`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000459035', 'lastCheck': '3.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:01,909::task::595::Storage.TaskManager.Task::(_updateState) Task=`a073fa5e-aad4-4f76-bbc1-2bc74fa0dc33`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:01,909::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:01,909::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:01,910::task::993::Storage.TaskManager.Task::(_decref) Task=`a073fa5e-aad4-4f76-bbc1-2bc74fa0dc33`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:04:01,937::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:03,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:03,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205155 s, 49.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:04,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:04,701::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:04:04,737::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:04:04,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f642414-cdff-441d-be38-848982489d79`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:04,777::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:04,778::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:04:04,791::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:04,791::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f642414-cdff-441d-be38-848982489d79`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:04,791::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f642414-cdff-441d-be38-848982489d79`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:04,792::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:04,792::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:04,792::task::993::Storage.TaskManager.Task::(_decref) Task=`0f642414-cdff-441d-be38-848982489d79`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:04:04,803::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:04:04,804::task::595::Storage.TaskManager.Task::(_updateState) Task=`bee43f59-2d4d-439d-9daf-aad11f230a7f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:04:04,804::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:04:04,805::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`289ed647-4b6b-4ae1-8170-873b58fc6c0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:04:04,805::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:04:04,806::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:04:04,806::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`289ed647-4b6b-4ae1-8170-873b58fc6c0f`::Granted request >Thread-34::DEBUG::2014-08-11 14:04:04,806::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bee43f59-2d4d-439d-9daf-aad11f230a7f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:04:04,807::task::993::Storage.TaskManager.Task::(_decref) Task=`bee43f59-2d4d-439d-9daf-aad11f230a7f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:04:04,827::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:04,829::task::1191::Storage.TaskManager.Task::(prepare) Task=`bee43f59-2d4d-439d-9daf-aad11f230a7f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:04,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`bee43f59-2d4d-439d-9daf-aad11f230a7f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:04:04,831::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:04:04,832::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:04:04,834::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:04:04,836::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:04:04,837::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:04:04,838::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:04:04,840::task::993::Storage.TaskManager.Task::(_decref) Task=`bee43f59-2d4d-439d-9daf-aad11f230a7f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:04:05,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:05,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131779 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:04:07,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc68386f-4d29-4196-8295-770859b0066c`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:04:07,247::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:04:07,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:07,248::__init__::232::IOProcess::(_processLogs) (4293) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:04:07,249::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:07,250::__init__::232::IOProcess::(_processLogs) (4294) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:04:07,250::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:04:07,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc68386f-4d29-4196-8295-770859b0066c`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:04:07,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc68386f-4d29-4196-8295-770859b0066c`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:04:07,251::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:04:07,251::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:04:07,251::task::993::Storage.TaskManager.Task::(_decref) Task=`bc68386f-4d29-4196-8295-770859b0066c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:04:07,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:07,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132454 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:08,635::__init__::232::IOProcess::(_processLogs) (4295) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:04:08,636::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:04:08,654::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420002 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:08,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:08,655::__init__::232::IOProcess::(_processLogs) (4296) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:04:08,656::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:08,657::__init__::232::IOProcess::(_processLogs) (4297) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:08,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:08,658::__init__::232::IOProcess::(_processLogs) (4298) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:08,659::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:08,659::__init__::232::IOProcess::(_processLogs) (4299) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:08,660::__init__::232::IOProcess::(_processLogs) (4299) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:09,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:09,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:09,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:09,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:09,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013236 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:11,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:12,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131864 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:14,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:14,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131898 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:14,855::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:04:14,856::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c9a6b93-17e5-4fa7-aa4e-2f115af846ed`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:14,872::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:14,882::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:14,883::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:14,884::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:04:14,885::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:14,886::task::1191::Storage.TaskManager.Task::(prepare) Task=`7c9a6b93-17e5-4fa7-aa4e-2f115af846ed`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:14,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c9a6b93-17e5-4fa7-aa4e-2f115af846ed`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:14,886::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:14,887::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:14,887::task::993::Storage.TaskManager.Task::(_decref) Task=`7c9a6b93-17e5-4fa7-aa4e-2f115af846ed`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:04:14,897::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:04:14,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`5286c020-a76e-4b75-a21e-315197e016ec`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:04:14,898::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:04:14,899::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`03340c58-85f8-4756-8970-221e336f1b6f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:04:14,899::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:04:14,899::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:04:14,900::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`03340c58-85f8-4756-8970-221e336f1b6f`::Granted request >Thread-34::DEBUG::2014-08-11 14:04:14,900::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5286c020-a76e-4b75-a21e-315197e016ec`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:04:14,900::task::993::Storage.TaskManager.Task::(_decref) Task=`5286c020-a76e-4b75-a21e-315197e016ec`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:04:14,920::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:14,922::task::1191::Storage.TaskManager.Task::(prepare) Task=`5286c020-a76e-4b75-a21e-315197e016ec`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:14,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`5286c020-a76e-4b75-a21e-315197e016ec`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:04:14,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:04:14,925::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:04:14,928::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:04:14,929::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:04:14,930::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:04:14,931::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:04:14,932::task::993::Storage.TaskManager.Task::(_decref) Task=`5286c020-a76e-4b75-a21e-315197e016ec`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:04:16,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:16,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132861 s, 77.1 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:17,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b8e807c-8af6-4e9b-8f22-9d1b0cc1f099`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:17,064::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:04:17,065::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420002', 'lastCheck': '8.4', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:17,065::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b8e807c-8af6-4e9b-8f22-9d1b0cc1f099`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420002', 'lastCheck': '8.4', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:17,065::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b8e807c-8af6-4e9b-8f22-9d1b0cc1f099`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:17,066::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:17,066::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:17,066::task::993::Storage.TaskManager.Task::(_decref) Task=`8b8e807c-8af6-4e9b-8f22-9d1b0cc1f099`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:04:17,087::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:18,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:18,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131235 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:18,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:18,662::__init__::232::IOProcess::(_processLogs) (4300) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:04:18,662::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:04:18,679::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423812 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:18,680::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:18,681::__init__::232::IOProcess::(_processLogs) (4301) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:04:18,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:18,682::__init__::232::IOProcess::(_processLogs) (4302) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:18,683::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:18,683::__init__::232::IOProcess::(_processLogs) (4303) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:18,684::__init__::232::IOProcess::(_processLogs) (4303) Queuing response >Thread-14::DEBUG::2014-08-11 14:04:18,685::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:18,685::__init__::232::IOProcess::(_processLogs) (4304) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:18,685::__init__::232::IOProcess::(_processLogs) (4304) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:19,892::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:19,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:19,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:20,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:20,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133154 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:22,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:22,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131492 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:24,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:24,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131903 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:24,948::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:04:24,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f249bf5-c647-4976-a3b6-131448862f90`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:24,973::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:24,979::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:24,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:24,982::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:04:24,986::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:24,987::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f249bf5-c647-4976-a3b6-131448862f90`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:24,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f249bf5-c647-4976-a3b6-131448862f90`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:24,987::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:24,987::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:24,988::task::993::Storage.TaskManager.Task::(_decref) Task=`0f249bf5-c647-4976-a3b6-131448862f90`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:04:24,998::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:04:24,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`a267d7d6-778b-4299-af7c-097051aa5902`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:04:24,999::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:04:25,000::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7e8aaeb3-4725-4d2e-87ce-4ef18f3b9ffe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:04:25,000::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:04:25,001::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:04:25,001::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7e8aaeb3-4725-4d2e-87ce-4ef18f3b9ffe`::Granted request >Thread-34::DEBUG::2014-08-11 14:04:25,002::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a267d7d6-778b-4299-af7c-097051aa5902`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:04:25,002::task::993::Storage.TaskManager.Task::(_decref) Task=`a267d7d6-778b-4299-af7c-097051aa5902`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:04:25,024::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:25,025::task::1191::Storage.TaskManager.Task::(prepare) Task=`a267d7d6-778b-4299-af7c-097051aa5902`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:25,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`a267d7d6-778b-4299-af7c-097051aa5902`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:04:25,027::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:04:25,029::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:04:25,031::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:04:25,033::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:04:25,034::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:04:25,035::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:04:25,038::task::993::Storage.TaskManager.Task::(_decref) Task=`a267d7d6-778b-4299-af7c-097051aa5902`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:04:26,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:26,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133384 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:28,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:28,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132735 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:28,687::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:28,688::__init__::232::IOProcess::(_processLogs) (4305) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:04:28,688::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:04:28,707::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000655585 s, 694 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:28,708::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:28,709::__init__::232::IOProcess::(_processLogs) (4306) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:04:28,710::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:28,710::__init__::232::IOProcess::(_processLogs) (4307) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:28,711::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:28,711::__init__::232::IOProcess::(_processLogs) (4308) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:28,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:28,713::__init__::232::IOProcess::(_processLogs) (4309) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:28,713::__init__::232::IOProcess::(_processLogs) (4309) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:29,985::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:29,987::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:30,075::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:30,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:30,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134171 s, 76.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:32,175::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fbfd70a-bc3b-4270-a97d-11bc56c6ed93`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:32,175::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:04:32,175::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000655585', 'lastCheck': '3.5', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:32,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`3fbfd70a-bc3b-4270-a97d-11bc56c6ed93`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000655585', 'lastCheck': '3.5', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:32,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fbfd70a-bc3b-4270-a97d-11bc56c6ed93`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:32,176::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:32,176::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:32,176::task::993::Storage.TaskManager.Task::(_decref) Task=`3fbfd70a-bc3b-4270-a97d-11bc56c6ed93`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:04:32,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:32,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:32,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013169 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:34,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:34,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132188 s, 77.5 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:35,055::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:04:35,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`24efc102-be35-4343-b456-44aa0e1c4f42`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:35,073::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:35,080::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:35,082::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:35,083::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:04:35,087::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:35,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`24efc102-be35-4343-b456-44aa0e1c4f42`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:35,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`24efc102-be35-4343-b456-44aa0e1c4f42`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:35,088::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:35,088::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:35,088::task::993::Storage.TaskManager.Task::(_decref) Task=`24efc102-be35-4343-b456-44aa0e1c4f42`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:04:35,099::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:04:35,099::task::595::Storage.TaskManager.Task::(_updateState) Task=`e22c8df6-13f8-444d-85d1-08c9bf860b34`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:04:35,100::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:04:35,100::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4bc8f286-a73a-4234-a7ea-bf668f63f178`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:04:35,101::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:04:35,101::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:04:35,101::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4bc8f286-a73a-4234-a7ea-bf668f63f178`::Granted request >Thread-34::DEBUG::2014-08-11 14:04:35,102::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e22c8df6-13f8-444d-85d1-08c9bf860b34`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:04:35,102::task::993::Storage.TaskManager.Task::(_decref) Task=`e22c8df6-13f8-444d-85d1-08c9bf860b34`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:04:35,124::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:35,126::task::1191::Storage.TaskManager.Task::(prepare) Task=`e22c8df6-13f8-444d-85d1-08c9bf860b34`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:35,127::task::595::Storage.TaskManager.Task::(_updateState) Task=`e22c8df6-13f8-444d-85d1-08c9bf860b34`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:04:35,128::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:04:35,129::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:04:35,131::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:04:35,133::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:04:35,134::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:04:35,135::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:04:35,136::task::993::Storage.TaskManager.Task::(_decref) Task=`e22c8df6-13f8-444d-85d1-08c9bf860b34`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:04:36,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:36,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132663 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:38,715::__init__::232::IOProcess::(_processLogs) (4310) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:04:38,716::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:04:38,733::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425605 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:38,734::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:38,735::__init__::232::IOProcess::(_processLogs) (4311) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:04:38,736::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:38,736::__init__::232::IOProcess::(_processLogs) (4312) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:38,737::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:38,737::__init__::232::IOProcess::(_processLogs) (4313) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:38,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:38,739::__init__::232::IOProcess::(_processLogs) (4314) Queuing response >Dummy-33::DEBUG::2014-08-11 14:04:38,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:38,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134126 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:40,087::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:40,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:40,174::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:40,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:40,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131431 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:43,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:43,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013137 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:45,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:45,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132381 s, 77.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:45,150::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:04:45,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`c43db58b-547b-44e4-8115-978a8218319e`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:45,152::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:04:45,165::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:45,165::task::1191::Storage.TaskManager.Task::(prepare) Task=`c43db58b-547b-44e4-8115-978a8218319e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:45,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`c43db58b-547b-44e4-8115-978a8218319e`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:45,165::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:45,166::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:45,166::task::993::Storage.TaskManager.Task::(_decref) Task=`c43db58b-547b-44e4-8115-978a8218319e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:04:45,176::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:04:45,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cd55979-f02b-44ae-8e68-d9345a9e9118`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:04:45,177::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:04:45,181::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`44e64398-04a2-41d4-b455-944ff693b37c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:45,183::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:04:45,183::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:04:45,185::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:04:45,186::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`44e64398-04a2-41d4-b455-944ff693b37c`::Granted request >Thread-34::DEBUG::2014-08-11 14:04:45,189::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2cd55979-f02b-44ae-8e68-d9345a9e9118`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:45,189::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:04:45,190::task::993::Storage.TaskManager.Task::(_decref) Task=`2cd55979-f02b-44ae-8e68-d9345a9e9118`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:45,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:04:45,220::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:45,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`2cd55979-f02b-44ae-8e68-d9345a9e9118`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:45,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cd55979-f02b-44ae-8e68-d9345a9e9118`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:04:45,224::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:04:45,225::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:04:45,227::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:04:45,229::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:04:45,230::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:04:45,231::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:04:45,233::task::993::Storage.TaskManager.Task::(_decref) Task=`2cd55979-f02b-44ae-8e68-d9345a9e9118`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:04:47,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:47,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131347 s, 78.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:47,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`d37024dc-f24c-45f5-91ec-73e05c944606`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:47,293::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:04:47,293::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425605', 'lastCheck': '8.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:47,293::task::1191::Storage.TaskManager.Task::(prepare) Task=`d37024dc-f24c-45f5-91ec-73e05c944606`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425605', 'lastCheck': '8.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:04:47,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`d37024dc-f24c-45f5-91ec-73e05c944606`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:47,294::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:47,294::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:47,294::task::993::Storage.TaskManager.Task::(_decref) Task=`d37024dc-f24c-45f5-91ec-73e05c944606`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:04:47,314::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:04:48,740::__init__::232::IOProcess::(_processLogs) (4315) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:04:48,741::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:04:48,758::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419585 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:48,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:48,760::__init__::232::IOProcess::(_processLogs) (4316) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:04:48,761::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:48,761::__init__::232::IOProcess::(_processLogs) (4317) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:48,762::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:48,763::__init__::232::IOProcess::(_processLogs) (4318) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:48,763::__init__::232::IOProcess::(_processLogs) (4318) Queuing response >Thread-14::DEBUG::2014-08-11 14:04:48,764::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:48,764::__init__::232::IOProcess::(_processLogs) (4319) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:48,764::__init__::232::IOProcess::(_processLogs) (4319) Queuing response >Dummy-33::DEBUG::2014-08-11 14:04:49,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:49,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132315 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:50,208::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:50,210::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:50,274::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:04:51,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:51,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013238 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:53,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:53,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131885 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:04:55,247::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:04:55,248::task::595::Storage.TaskManager.Task::(_updateState) Task=`813626c6-ff25-4223-a927-5848a6a4f93f`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:04:55,273::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:55,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:55,283::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:04:55,284::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:04:55,286::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:55,287::task::1191::Storage.TaskManager.Task::(prepare) Task=`813626c6-ff25-4223-a927-5848a6a4f93f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:04:55,287::task::595::Storage.TaskManager.Task::(_updateState) Task=`813626c6-ff25-4223-a927-5848a6a4f93f`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:04:55,287::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:04:55,287::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:04:55,287::task::993::Storage.TaskManager.Task::(_decref) Task=`813626c6-ff25-4223-a927-5848a6a4f93f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:04:55,298::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:04:55,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a904d91-dcdb-4b6e-a20c-2aaa659d3094`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:04:55,299::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:04:55,300::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f50ebeee-9082-41cb-9045-f7644524710b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:04:55,300::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:04:55,300::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:04:55,300::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f50ebeee-9082-41cb-9045-f7644524710b`::Granted request >Thread-34::DEBUG::2014-08-11 14:04:55,301::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8a904d91-dcdb-4b6e-a20c-2aaa659d3094`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:04:55,301::task::993::Storage.TaskManager.Task::(_decref) Task=`8a904d91-dcdb-4b6e-a20c-2aaa659d3094`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:04:55,322::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:04:55,324::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a904d91-dcdb-4b6e-a20c-2aaa659d3094`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Dummy-33::DEBUG::2014-08-11 14:04:55,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::DEBUG::2014-08-11 14:04:55,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a904d91-dcdb-4b6e-a20c-2aaa659d3094`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:04:55,326::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:04:55,350::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:04:55,355::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:04:55,358::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:04:55,360::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:04:55,361::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:04:55,365::task::993::Storage.TaskManager.Task::(_decref) Task=`8a904d91-dcdb-4b6e-a20c-2aaa659d3094`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:04:55,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0372657 s, 27.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:04:57,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:57,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132876 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:58,766::__init__::232::IOProcess::(_processLogs) (4320) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:04:58,767::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:04:58,786::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000440271 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:04:58,788::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:58,788::__init__::232::IOProcess::(_processLogs) (4321) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:04:58,789::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:58,790::__init__::232::IOProcess::(_processLogs) (4322) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:58,791::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:58,791::__init__::232::IOProcess::(_processLogs) (4323) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:04:58,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:04:58,792::__init__::232::IOProcess::(_processLogs) (4324) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:04:59,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:04:59,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133604 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:00,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:00,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:00,374::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:01,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:01,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133976 s, 76.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:02,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`8465782a-377f-4158-a77e-f5a32929c5ed`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:02,409::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:05:02,411::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000440271', 'lastCheck': '3.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:02,413::task::1191::Storage.TaskManager.Task::(prepare) Task=`8465782a-377f-4158-a77e-f5a32929c5ed`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000440271', 'lastCheck': '3.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:02,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`8465782a-377f-4158-a77e-f5a32929c5ed`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:02,415::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:02,416::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:02,417::task::993::Storage.TaskManager.Task::(_decref) Task=`8465782a-377f-4158-a77e-f5a32929c5ed`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:05:02,459::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:03,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:03,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132237 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:05,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:05,383::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:05,384::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:05:05,388::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:05:05,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`b875a858-ea45-4269-83a8-72112a16ba68`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:05,389::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:05:05,402::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:05,402::task::1191::Storage.TaskManager.Task::(prepare) Task=`b875a858-ea45-4269-83a8-72112a16ba68`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:05,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`b875a858-ea45-4269-83a8-72112a16ba68`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:05,403::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:05,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:05,403::task::993::Storage.TaskManager.Task::(_decref) Task=`b875a858-ea45-4269-83a8-72112a16ba68`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:05:05,414::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:05:05,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`9813caca-ae0c-464d-97e7-ab2fd93ba246`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:05:05,415::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:05:05,416::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4faa1bbe-50f1-4d32-8126-219135bc3348`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:05:05,416::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:05:05,416::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:05:05,417::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4faa1bbe-50f1-4d32-8126-219135bc3348`::Granted request >Thread-34::DEBUG::2014-08-11 14:05:05,417::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9813caca-ae0c-464d-97e7-ab2fd93ba246`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:05:05,417::task::993::Storage.TaskManager.Task::(_decref) Task=`9813caca-ae0c-464d-97e7-ab2fd93ba246`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:05:05,437::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:05,438::task::1191::Storage.TaskManager.Task::(prepare) Task=`9813caca-ae0c-464d-97e7-ab2fd93ba246`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:05,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`9813caca-ae0c-464d-97e7-ab2fd93ba246`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:05:05,438::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:05:05,438::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:05:05,439::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:05:05,439::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:05:05,439::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:05:05,439::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:05:05,440::task::993::Storage.TaskManager.Task::(_decref) Task=`9813caca-ae0c-464d-97e7-ab2fd93ba246`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:05:05,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:05,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132318 s, 77.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:05:07,367::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e17ec5a-b093-454e-a79c-d3af150e94a4`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:05:07,367::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:05:07,368::__init__::232::IOProcess::(_processLogs) (4325) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:05:07,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:07,370::__init__::232::IOProcess::(_processLogs) (4326) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:05:07,371::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:05:07,371::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e17ec5a-b093-454e-a79c-d3af150e94a4`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:05:07,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e17ec5a-b093-454e-a79c-d3af150e94a4`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:05:07,372::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:05:07,372::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:05:07,372::task::993::Storage.TaskManager.Task::(_decref) Task=`0e17ec5a-b093-454e-a79c-d3af150e94a4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:05:07,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:07,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131951 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:08,794::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:08,795::__init__::232::IOProcess::(_processLogs) (4327) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:05:08,795::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:05:08,813::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00044435 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:08,814::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:08,814::__init__::232::IOProcess::(_processLogs) (4328) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:08,815::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:08,816::__init__::232::IOProcess::(_processLogs) (4329) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:08,817::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:08,817::__init__::232::IOProcess::(_processLogs) (4330) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:08,818::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:08,818::__init__::232::IOProcess::(_processLogs) (4331) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:05:09,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:09,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131872 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:10,391::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:10,392::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:10,474::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:11,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:11,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013278 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:13,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:14,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132279 s, 77.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:15,452::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-81::DEBUG::2014-08-11 14:05:15,453::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d23dee1-496f-4629-aa7a-e063760bc6f1`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:15,473::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:15,480::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:15,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:15,483::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:05:15,486::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:15,486::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d23dee1-496f-4629-aa7a-e063760bc6f1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:15,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d23dee1-496f-4629-aa7a-e063760bc6f1`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:15,487::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:15,487::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:15,487::task::993::Storage.TaskManager.Task::(_decref) Task=`3d23dee1-496f-4629-aa7a-e063760bc6f1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:05:15,498::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 14:05:15,499::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f43e552-5577-472e-83d7-9767cc67c921`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:05:15,499::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:05:15,500::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`052f4e49-ab5f-41b9-ba02-754f471116fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:05:15,501::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:05:15,501::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:05:15,501::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`052f4e49-ab5f-41b9-ba02-754f471116fa`::Granted request >Thread-34::DEBUG::2014-08-11 14:05:15,502::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0f43e552-5577-472e-83d7-9767cc67c921`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:05:15,502::task::993::Storage.TaskManager.Task::(_decref) Task=`0f43e552-5577-472e-83d7-9767cc67c921`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:05:15,522::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:15,522::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f43e552-5577-472e-83d7-9767cc67c921`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:15,522::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f43e552-5577-472e-83d7-9767cc67c921`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:05:15,523::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:05:15,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:05:15,523::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:05:15,523::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:05:15,524::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:05:15,524::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:05:15,524::task::993::Storage.TaskManager.Task::(_decref) Task=`0f43e552-5577-472e-83d7-9767cc67c921`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:05:16,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:16,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131214 s, 78.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:17,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`61e49a75-8681-4268-9f68-801245cbaecb`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:17,571::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:05:17,571::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00044435', 'lastCheck': '8.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:17,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`61e49a75-8681-4268-9f68-801245cbaecb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00044435', 'lastCheck': '8.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:17,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`61e49a75-8681-4268-9f68-801245cbaecb`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:17,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:17,572::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:17,573::task::993::Storage.TaskManager.Task::(_decref) Task=`61e49a75-8681-4268-9f68-801245cbaecb`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:05:17,591::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:18,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:18,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133104 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:18,820::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:18,821::__init__::232::IOProcess::(_processLogs) (4332) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:05:18,821::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:05:18,839::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424949 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:18,840::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:18,841::__init__::232::IOProcess::(_processLogs) (4333) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:18,842::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:18,842::__init__::232::IOProcess::(_processLogs) (4334) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:18,843::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:18,843::__init__::232::IOProcess::(_processLogs) (4335) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:18,844::__init__::232::IOProcess::(_processLogs) (4335) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:18,845::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:18,845::__init__::232::IOProcess::(_processLogs) (4336) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:05:20,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:20,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132365 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:20,491::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:20,492::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:20,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:22,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:22,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132013 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:24,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:24,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131243 s, 78.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:25,537::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:05:25,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`940b810a-1710-4305-8f2c-2d12b6d3ff7f`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:25,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:25,580::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:25,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:25,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:05:25,587::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:25,588::task::1191::Storage.TaskManager.Task::(prepare) Task=`940b810a-1710-4305-8f2c-2d12b6d3ff7f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:25,588::task::595::Storage.TaskManager.Task::(_updateState) Task=`940b810a-1710-4305-8f2c-2d12b6d3ff7f`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:25,588::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:25,588::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:25,589::task::993::Storage.TaskManager.Task::(_decref) Task=`940b810a-1710-4305-8f2c-2d12b6d3ff7f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:05:25,600::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:05:25,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`04100d4b-a19c-4df8-a7b8-62950059877b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:05:25,601::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:05:25,602::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce1efe7f-a178-4655-a7bf-853d5f9d9229`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:05:25,602::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:05:25,603::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:05:25,603::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce1efe7f-a178-4655-a7bf-853d5f9d9229`::Granted request >Thread-34::DEBUG::2014-08-11 14:05:25,604::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`04100d4b-a19c-4df8-a7b8-62950059877b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:05:25,604::task::993::Storage.TaskManager.Task::(_decref) Task=`04100d4b-a19c-4df8-a7b8-62950059877b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:05:25,624::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:25,624::task::1191::Storage.TaskManager.Task::(prepare) Task=`04100d4b-a19c-4df8-a7b8-62950059877b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:25,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`04100d4b-a19c-4df8-a7b8-62950059877b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:05:25,625::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:05:25,625::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:05:25,625::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:05:25,626::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:05:25,626::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:05:25,626::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:05:25,626::task::993::Storage.TaskManager.Task::(_decref) Task=`04100d4b-a19c-4df8-a7b8-62950059877b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:05:26,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:26,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132418 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:28,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:28,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131978 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:28,847::__init__::232::IOProcess::(_processLogs) (4337) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:28,848::__init__::232::IOProcess::(_processLogs) DEBUG|(4337) Queuing response >Thread-16::DEBUG::2014-08-11 14:05:28,848::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:05:28,867::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421299 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:28,868::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:28,868::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:28,868::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:28,869::__init__::232::IOProcess::(_processLogs) (4338) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:28,869::__init__::232::IOProcess::(_processLogs) (4338) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:28,871::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:28,871::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:28,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:28,872::__init__::232::IOProcess::(_processLogs) (4339) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:28,872::__init__::232::IOProcess::(_processLogs) (4339) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:28,873::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:28,873::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:28,874::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:28,874::__init__::232::IOProcess::(_processLogs) (4340) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:28,874::__init__::232::IOProcess::(_processLogs) (4340) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:28,875::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:28,876::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:28,876::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:28,877::__init__::232::IOProcess::(_processLogs) (4341) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:28,877::__init__::232::IOProcess::(_processLogs) (4341) Queuing response >Dummy-33::DEBUG::2014-08-11 14:05:30,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:30,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133543 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:30,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:30,592::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:30,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:32,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:32,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130725 s, 78.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:32,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`4492e020-cc1b-4cd8-ac50-6fb6fd81ea2d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:32,689::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:05:32,690::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421299', 'lastCheck': '3.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:32,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`4492e020-cc1b-4cd8-ac50-6fb6fd81ea2d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421299', 'lastCheck': '3.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:32,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`4492e020-cc1b-4cd8-ac50-6fb6fd81ea2d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:32,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:32,695::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:32,696::task::993::Storage.TaskManager.Task::(_decref) Task=`4492e020-cc1b-4cd8-ac50-6fb6fd81ea2d`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:05:32,742::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:34,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:34,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132125 s, 77.5 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:35,650::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:05:35,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a77b5c7-899b-4517-88b4-7baf71a5f267`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:35,674::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:35,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:35,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:35,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:05:35,688::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:35,689::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a77b5c7-899b-4517-88b4-7baf71a5f267`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:35,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a77b5c7-899b-4517-88b4-7baf71a5f267`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:35,689::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:35,689::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:35,689::task::993::Storage.TaskManager.Task::(_decref) Task=`1a77b5c7-899b-4517-88b4-7baf71a5f267`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:05:35,700::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:05:35,701::task::595::Storage.TaskManager.Task::(_updateState) Task=`46aaa629-335d-4ca2-888b-99de8549f230`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:05:35,701::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:05:35,702::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dd35f089-e13c-4dc9-b54e-7a0fa017ca99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:05:35,702::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:05:35,702::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:05:35,703::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dd35f089-e13c-4dc9-b54e-7a0fa017ca99`::Granted request >Thread-34::DEBUG::2014-08-11 14:05:35,703::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`46aaa629-335d-4ca2-888b-99de8549f230`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:05:35,703::task::993::Storage.TaskManager.Task::(_decref) Task=`46aaa629-335d-4ca2-888b-99de8549f230`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:05:35,723::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:35,723::task::1191::Storage.TaskManager.Task::(prepare) Task=`46aaa629-335d-4ca2-888b-99de8549f230`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:35,724::task::595::Storage.TaskManager.Task::(_updateState) Task=`46aaa629-335d-4ca2-888b-99de8549f230`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:05:35,724::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:05:35,724::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:05:35,724::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:05:35,725::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:05:35,725::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:05:35,725::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:05:35,725::task::993::Storage.TaskManager.Task::(_decref) Task=`46aaa629-335d-4ca2-888b-99de8549f230`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:05:36,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:36,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131438 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:38,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:38,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132005 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:38,878::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:38,878::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:38,878::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:38,879::__init__::232::IOProcess::(_processLogs) (4342) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:38,879::__init__::232::IOProcess::(_processLogs) (4342) Queuing response >Thread-16::DEBUG::2014-08-11 14:05:38,880::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:05:38,897::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415214 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:38,975::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:38,976::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:38,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:38,976::__init__::232::IOProcess::(_processLogs) (4343) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:38,976::__init__::232::IOProcess::(_processLogs) (4343) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:38,978::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:38,978::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:38,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:38,979::__init__::232::IOProcess::(_processLogs) (4344) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:38,979::__init__::232::IOProcess::(_processLogs) (4344) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:38,980::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:38,980::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:38,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:38,981::__init__::232::IOProcess::(_processLogs) (4345) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:38,981::__init__::232::IOProcess::(_processLogs) (4345) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:38,982::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:38,982::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:38,983::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:38,984::__init__::232::IOProcess::(_processLogs) (4346) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:38,984::__init__::232::IOProcess::(_processLogs) (4346) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:40,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:40,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:40,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:40,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:40,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131382 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:42,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:42,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132085 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:44,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:44,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131808 s, 77.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:45,737::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:05:45,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`b140e99a-6ce2-4bfe-b785-c65cd96751f4`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:45,775::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:45,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:45,784::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:45,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:05:45,788::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:45,789::task::1191::Storage.TaskManager.Task::(prepare) Task=`b140e99a-6ce2-4bfe-b785-c65cd96751f4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:45,789::task::595::Storage.TaskManager.Task::(_updateState) Task=`b140e99a-6ce2-4bfe-b785-c65cd96751f4`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:45,789::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:45,789::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:45,789::task::993::Storage.TaskManager.Task::(_decref) Task=`b140e99a-6ce2-4bfe-b785-c65cd96751f4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:05:45,799::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:05:45,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`8cb59e07-9c1f-4e32-a30a-fc7c50f90943`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:05:45,800::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:05:45,801::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6356ba6b-c4dc-4a3b-a666-d46492ef4469`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:05:45,801::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:05:45,802::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:05:45,802::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6356ba6b-c4dc-4a3b-a666-d46492ef4469`::Granted request >Thread-34::DEBUG::2014-08-11 14:05:45,803::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8cb59e07-9c1f-4e32-a30a-fc7c50f90943`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:05:45,803::task::993::Storage.TaskManager.Task::(_decref) Task=`8cb59e07-9c1f-4e32-a30a-fc7c50f90943`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:05:45,823::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:45,823::task::1191::Storage.TaskManager.Task::(prepare) Task=`8cb59e07-9c1f-4e32-a30a-fc7c50f90943`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:45,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`8cb59e07-9c1f-4e32-a30a-fc7c50f90943`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:05:45,823::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:05:45,824::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:05:45,824::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:05:45,825::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:05:45,825::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:05:45,825::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:05:45,825::task::993::Storage.TaskManager.Task::(_decref) Task=`8cb59e07-9c1f-4e32-a30a-fc7c50f90943`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:05:47,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:47,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137711 s, 74.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:47,871::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0be4bc3-a911-4c84-a2e0-c438fb70ee3d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:47,872::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:05:47,872::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415214', 'lastCheck': '8.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:47,872::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0be4bc3-a911-4c84-a2e0-c438fb70ee3d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415214', 'lastCheck': '8.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:05:47,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0be4bc3-a911-4c84-a2e0-c438fb70ee3d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:47,873::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:47,873::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:47,873::task::993::Storage.TaskManager.Task::(_decref) Task=`a0be4bc3-a911-4c84-a2e0-c438fb70ee3d`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:05:47,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:05:48,985::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:48,985::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:48,985::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:48,986::__init__::232::IOProcess::(_processLogs) (4347) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:48,986::__init__::232::IOProcess::(_processLogs) (4347) Queuing response >Thread-16::DEBUG::2014-08-11 14:05:48,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:05:49,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000429966 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:49,005::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:49,006::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:49,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:49,006::__init__::232::IOProcess::(_processLogs) (4348) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:49,006::__init__::232::IOProcess::(_processLogs) (4348) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:49,007::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:49,008::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:49,009::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:49,009::__init__::232::IOProcess::(_processLogs) (4349) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:49,009::__init__::232::IOProcess::(_processLogs) (4349) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:49,009::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:49,010::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:49,011::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:49,011::__init__::232::IOProcess::(_processLogs) (4350) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:49,011::__init__::232::IOProcess::(_processLogs) (4350) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:49,012::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:49,012::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:49,012::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:49,013::__init__::232::IOProcess::(_processLogs) (4351) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:49,014::__init__::232::IOProcess::(_processLogs) (4351) Queuing response >Dummy-33::DEBUG::2014-08-11 14:05:49,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:49,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132537 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:50,789::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:50,791::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:50,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:05:51,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:51,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132311 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:53,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:53,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132275 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:05:55,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:55,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013247 s, 77.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:05:55,839::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:05:55,840::task::595::Storage.TaskManager.Task::(_updateState) Task=`92a2a5a5-7f50-448c-ae81-3945e71f3eb1`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:05:55,877::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:55,884::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:55,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:05:55,887::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:05:55,890::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:55,891::task::1191::Storage.TaskManager.Task::(prepare) Task=`92a2a5a5-7f50-448c-ae81-3945e71f3eb1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:05:55,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`92a2a5a5-7f50-448c-ae81-3945e71f3eb1`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:05:55,891::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:05:55,891::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:05:55,891::task::993::Storage.TaskManager.Task::(_decref) Task=`92a2a5a5-7f50-448c-ae81-3945e71f3eb1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:05:55,906::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:05:55,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`0439f803-d9c9-4fa5-81a4-a6bef8b56036`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:05:55,907::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:05:55,908::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a466d8a9-2c30-4cfc-bbb7-78f1bfc5b0e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:05:55,908::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:05:55,908::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:05:55,909::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a466d8a9-2c30-4cfc-bbb7-78f1bfc5b0e9`::Granted request >Thread-34::DEBUG::2014-08-11 14:05:55,909::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0439f803-d9c9-4fa5-81a4-a6bef8b56036`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:05:55,909::task::993::Storage.TaskManager.Task::(_decref) Task=`0439f803-d9c9-4fa5-81a4-a6bef8b56036`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:05:55,929::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:55,930::task::1191::Storage.TaskManager.Task::(prepare) Task=`0439f803-d9c9-4fa5-81a4-a6bef8b56036`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:05:55,930::task::595::Storage.TaskManager.Task::(_updateState) Task=`0439f803-d9c9-4fa5-81a4-a6bef8b56036`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:05:55,930::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:05:55,930::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:05:55,931::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:05:55,931::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:05:55,931::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:05:55,931::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:05:55,932::task::993::Storage.TaskManager.Task::(_decref) Task=`0439f803-d9c9-4fa5-81a4-a6bef8b56036`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:05:57,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:57,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013152 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:59,014::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:59,015::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:59,015::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:59,015::__init__::232::IOProcess::(_processLogs) (4352) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:59,016::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4352) Queuing response >Thread-16::DEBUG::2014-08-11 14:05:59,017::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:05:59,035::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414659 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:05:59,036::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:59,037::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:59,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:59,037::__init__::232::IOProcess::(_processLogs) (4353) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:05:59,038::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4353) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:59,040::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:59,040::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:59,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:59,041::__init__::232::IOProcess::(_processLogs) (4354) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:59,041::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4354) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:59,042::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:59,043::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:59,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:59,043::__init__::232::IOProcess::(_processLogs) (4355) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:59,044::__init__::232::IOProcess::(_processLogs) (4355) Queuing response >Thread-14::DEBUG::2014-08-11 14:05:59,044::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:05:59,044::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:05:59,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:05:59,045::__init__::232::IOProcess::(_processLogs) (4356) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:05:59,045::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4356) Queuing response >Dummy-33::DEBUG::2014-08-11 14:05:59,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:05:59,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132167 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:00,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:00,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:00,980::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:01,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:01,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133453 s, 76.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:02,982::task::595::Storage.TaskManager.Task::(_updateState) Task=`401d425d-1927-4260-b0a0-f00f9dc13dc4`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:02,982::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:06:02,983::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414659', 'lastCheck': '3.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:02,983::task::1191::Storage.TaskManager.Task::(prepare) Task=`401d425d-1927-4260-b0a0-f00f9dc13dc4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414659', 'lastCheck': '3.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:02,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`401d425d-1927-4260-b0a0-f00f9dc13dc4`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:02,983::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:02,983::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:02,984::task::993::Storage.TaskManager.Task::(_decref) Task=`401d425d-1927-4260-b0a0-f00f9dc13dc4`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:06:03,003::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:03,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:03,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131372 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:05,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:05,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132249 s, 77.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:05,945::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:06:05,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`90294aae-0c0e-4b58-ad18-504d25914c8f`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:05,977::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:05,987::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:05,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:05,990::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:06:05,990::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:05,992::task::1191::Storage.TaskManager.Task::(prepare) Task=`90294aae-0c0e-4b58-ad18-504d25914c8f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:05,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`90294aae-0c0e-4b58-ad18-504d25914c8f`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:05,994::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:05,994::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:05,994::task::993::Storage.TaskManager.Task::(_decref) Task=`90294aae-0c0e-4b58-ad18-504d25914c8f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:06:06,004::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:06:06,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4fb985a-3ac2-473b-b31b-b5bce80cfa25`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:06:06,005::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:06:06,006::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bd0a24b9-09e3-4d71-b953-f066d2e7cdf5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:06:06,006::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:06:06,007::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:06:06,007::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bd0a24b9-09e3-4d71-b953-f066d2e7cdf5`::Granted request >Thread-34::DEBUG::2014-08-11 14:06:06,007::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e4fb985a-3ac2-473b-b31b-b5bce80cfa25`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:06:06,008::task::993::Storage.TaskManager.Task::(_decref) Task=`e4fb985a-3ac2-473b-b31b-b5bce80cfa25`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:06:06,027::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:06,028::task::1191::Storage.TaskManager.Task::(prepare) Task=`e4fb985a-3ac2-473b-b31b-b5bce80cfa25`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:06,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4fb985a-3ac2-473b-b31b-b5bce80cfa25`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:06:06,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:06:06,028::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:06:06,029::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:06:06,029::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:06:06,029::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:06:06,029::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:06:06,030::task::993::Storage.TaskManager.Task::(_decref) Task=`e4fb985a-3ac2-473b-b31b-b5bce80cfa25`::ref 0 aborting False >Thread-55::DEBUG::2014-08-11 14:06:07,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b81f36f-3fc5-4648-a00d-e82be069d9fa`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:06:07,484::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:06:07,485::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:07,485::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:07,486::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:07,486::__init__::232::IOProcess::(_processLogs) (4357) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:06:07,487::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4357) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:07,488::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:07,488::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:07,488::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:07,488::__init__::232::IOProcess::(_processLogs) (4358) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:06:07,489::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4358) Queuing response >Thread-55::INFO::2014-08-11 14:06:07,489::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:06:07,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b81f36f-3fc5-4648-a00d-e82be069d9fa`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:06:07,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b81f36f-3fc5-4648-a00d-e82be069d9fa`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:06:07,491::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:06:07,491::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:06:07,491::task::993::Storage.TaskManager.Task::(_decref) Task=`9b81f36f-3fc5-4648-a00d-e82be069d9fa`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:06:07,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:07,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013205 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:09,047::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:09,078::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:06:09,079::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:06:09,080::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:09,096::__init__::232::IOProcess::(_processLogs) (4359) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:09,097::__init__::232::IOProcess::(_processLogs) (4359) Queuing response >Thread-16::DEBUG::2014-08-11 14:06:09,098::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000866745 s, 525 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:09,099::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:09,099::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:09,100::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:09,100::__init__::232::IOProcess::(_processLogs) (4360) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:09,100::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4360) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:09,101::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:09,102::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:09,102::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:09,102::__init__::232::IOProcess::(_processLogs) (4361) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:09,102::__init__::232::IOProcess::(_processLogs) (4361) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:09,103::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:09,103::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:09,104::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:09,104::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4362) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:09,104::__init__::232::IOProcess::(_processLogs) (4362) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:09,105::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:09,105::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:09,105::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:09,106::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|(4363) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:09,106::__init__::232::IOProcess::(_processLogs) (4363) Queuing response >Dummy-33::DEBUG::2014-08-11 14:06:09,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:09,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131686 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:11,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:11,001::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:11,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:11,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:11,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131259 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:13,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:13,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132863 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:15,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:15,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132416 s, 77.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:16,044::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:06:16,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bb8ba1d-bd60-43c0-aef4-b0926526690e`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:16,078::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:16,085::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:16,087::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:16,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:06:16,090::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:16,091::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bb8ba1d-bd60-43c0-aef4-b0926526690e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:16,091::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bb8ba1d-bd60-43c0-aef4-b0926526690e`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:16,091::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:16,091::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:16,092::task::993::Storage.TaskManager.Task::(_decref) Task=`4bb8ba1d-bd60-43c0-aef4-b0926526690e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:06:16,102::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:06:16,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`cff0211d-e0af-4433-8315-1f6c96c21158`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:06:16,103::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:06:16,104::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f08ac530-e5cf-4fa3-896b-438a8808286f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:06:16,104::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:06:16,105::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:06:16,105::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f08ac530-e5cf-4fa3-896b-438a8808286f`::Granted request >Thread-34::DEBUG::2014-08-11 14:06:16,105::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cff0211d-e0af-4433-8315-1f6c96c21158`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:06:16,106::task::993::Storage.TaskManager.Task::(_decref) Task=`cff0211d-e0af-4433-8315-1f6c96c21158`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:06:16,126::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:16,126::task::1191::Storage.TaskManager.Task::(prepare) Task=`cff0211d-e0af-4433-8315-1f6c96c21158`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:16,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`cff0211d-e0af-4433-8315-1f6c96c21158`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:06:16,126::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:06:16,127::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:06:16,127::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:06:16,127::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:06:16,128::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:06:16,128::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:06:16,128::task::993::Storage.TaskManager.Task::(_decref) Task=`cff0211d-e0af-4433-8315-1f6c96c21158`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:06:17,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:17,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132532 s, 77.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:18,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3636b7a-5be3-456e-912b-9a9a71f8ccc1`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:18,104::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:06:18,104::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000866745', 'lastCheck': '9.0', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:18,104::task::1191::Storage.TaskManager.Task::(prepare) Task=`b3636b7a-5be3-456e-912b-9a9a71f8ccc1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000866745', 'lastCheck': '9.0', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:18,104::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3636b7a-5be3-456e-912b-9a9a71f8ccc1`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:18,105::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:18,105::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:18,105::task::993::Storage.TaskManager.Task::(_decref) Task=`b3636b7a-5be3-456e-912b-9a9a71f8ccc1`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:06:18,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:06:19,108::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:19,108::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:19,108::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:19,109::__init__::232::IOProcess::(_processLogs) (4364) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:19,109::__init__::232::IOProcess::(_processLogs) (4364) Queuing response >Thread-16::DEBUG::2014-08-11 14:06:19,110::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:06:19,127::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413961 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:19,128::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:19,128::__init__::232::IOProcess::(_processLogs) (4338) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:19,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:19,129::__init__::232::IOProcess::(_processLogs) (4365) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:19,130::__init__::232::IOProcess::(_processLogs) (4365) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:19,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:19,131::__init__::232::IOProcess::(_processLogs) (4366) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:19,132::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:19,132::__init__::232::IOProcess::(_processLogs) (4367) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:19,133::__init__::232::IOProcess::(_processLogs) (4367) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:19,134::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:19,134::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:19,135::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:19,136::__init__::232::IOProcess::(_processLogs) (4368) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:19,136::__init__::232::IOProcess::(_processLogs) (4368) Queuing response >Dummy-33::DEBUG::2014-08-11 14:06:19,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:19,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133529 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:21,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:21,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:21,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:21,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:21,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131783 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:23,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:24,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131171 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:26,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:26,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133252 s, 76.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:26,140::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:06:26,141::task::595::Storage.TaskManager.Task::(_updateState) Task=`60b29c76-1d0e-4052-ba65-efdc444d1af9`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:26,178::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:26,185::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:26,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:26,187::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:06:26,191::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:26,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`60b29c76-1d0e-4052-ba65-efdc444d1af9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:26,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`60b29c76-1d0e-4052-ba65-efdc444d1af9`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:26,192::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:26,192::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:26,193::task::993::Storage.TaskManager.Task::(_decref) Task=`60b29c76-1d0e-4052-ba65-efdc444d1af9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:06:26,206::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:06:26,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d8668c6-8b75-459d-9b1e-c04affb99bd4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:06:26,207::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:06:26,208::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4a55776-b6b0-44d0-801f-d67845e1a60c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:06:26,208::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:06:26,208::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:06:26,209::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4a55776-b6b0-44d0-801f-d67845e1a60c`::Granted request >Thread-34::DEBUG::2014-08-11 14:06:26,209::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2d8668c6-8b75-459d-9b1e-c04affb99bd4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:06:26,210::task::993::Storage.TaskManager.Task::(_decref) Task=`2d8668c6-8b75-459d-9b1e-c04affb99bd4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:06:26,229::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:26,230::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d8668c6-8b75-459d-9b1e-c04affb99bd4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:26,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d8668c6-8b75-459d-9b1e-c04affb99bd4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:06:26,230::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:06:26,230::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:06:26,231::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:06:26,231::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:06:26,231::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:06:26,231::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:06:26,232::task::993::Storage.TaskManager.Task::(_decref) Task=`2d8668c6-8b75-459d-9b1e-c04affb99bd4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:06:28,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:28,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013101 s, 78.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:29,137::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:29,138::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:29,139::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:06:29,139::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:06:29,140::__init__::232::IOProcess::(_processLogs) (4369) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:29,157::__init__::232::IOProcess::(_processLogs) (4369) Queuing response >Thread-16::DEBUG::2014-08-11 14:06:29,158::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000461451 s, 986 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:29,159::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:29,160::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:29,160::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:29,160::__init__::232::IOProcess::(_processLogs) (4370) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:29,161::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|(4370) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:29,162::__init__::232::IOProcess::(_processLogs) (4367) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:29,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:29,164::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:29,164::__init__::232::IOProcess::(_processLogs) (4371) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:29,164::__init__::232::IOProcess::(_processLogs) (4371) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:29,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:29,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:29,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:29,166::__init__::232::IOProcess::(_processLogs) (4372) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:29,166::__init__::232::IOProcess::(_processLogs) (4372) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:29,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:29,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:29,168::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:29,169::__init__::232::IOProcess::(_processLogs) (4373) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:29,169::__init__::232::IOProcess::(_processLogs) (4373) Queuing response >Dummy-33::DEBUG::2014-08-11 14:06:30,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:30,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132225 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:31,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:31,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:31,280::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:32,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:32,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0290111 s, 35.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:33,238::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a1d1865-a898-4d6f-b417-c0087eea46d8`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:33,238::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:06:33,239::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461451', 'lastCheck': '4.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:33,239::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a1d1865-a898-4d6f-b417-c0087eea46d8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461451', 'lastCheck': '4.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:33,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a1d1865-a898-4d6f-b417-c0087eea46d8`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:33,239::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:33,240::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:33,240::task::993::Storage.TaskManager.Task::(_decref) Task=`8a1d1865-a898-4d6f-b417-c0087eea46d8`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:06:33,260::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:34,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:34,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131974 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:36,245::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-81::DEBUG::2014-08-11 14:06:36,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`4232e343-3db6-4bb5-a4fc-e29ac97ccacb`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:36,279::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:36,289::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:36,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:36,293::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:06:36,294::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:36,295::task::1191::Storage.TaskManager.Task::(prepare) Task=`4232e343-3db6-4bb5-a4fc-e29ac97ccacb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:36,295::task::595::Storage.TaskManager.Task::(_updateState) Task=`4232e343-3db6-4bb5-a4fc-e29ac97ccacb`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:36,295::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:36,295::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:36,296::task::993::Storage.TaskManager.Task::(_decref) Task=`4232e343-3db6-4bb5-a4fc-e29ac97ccacb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:06:36,307::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 14:06:36,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`be0aa708-0fda-4e3c-8708-95ba0a1e0387`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:06:36,308::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:06:36,309::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b8c9de46-79cb-4357-9dd3-16b45edddc12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:06:36,309::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:06:36,309::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:06:36,310::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b8c9de46-79cb-4357-9dd3-16b45edddc12`::Granted request >Thread-34::DEBUG::2014-08-11 14:06:36,314::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`be0aa708-0fda-4e3c-8708-95ba0a1e0387`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:06:36,316::task::993::Storage.TaskManager.Task::(_decref) Task=`be0aa708-0fda-4e3c-8708-95ba0a1e0387`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:06:36,351::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:36,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`be0aa708-0fda-4e3c-8708-95ba0a1e0387`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:36,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`be0aa708-0fda-4e3c-8708-95ba0a1e0387`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:06:36,354::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:06:36,356::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:06:36,358::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:06:36,359::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:06:36,360::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:06:36,361::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:06:36,363::task::993::Storage.TaskManager.Task::(_decref) Task=`be0aa708-0fda-4e3c-8708-95ba0a1e0387`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:06:36,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:36,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131395 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:38,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:38,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013265 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:39,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:39,171::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:06:39,174::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:06:39,174::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:39,191::__init__::232::IOProcess::(_processLogs) (4374) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:39,192::__init__::232::IOProcess::(_processLogs) (4374) Queuing response >Thread-16::DEBUG::2014-08-11 14:06:39,193::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000500847 s, 908 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:39,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:39,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:39,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:39,195::__init__::232::IOProcess::(_processLogs) (4375) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:39,195::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4375) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:39,197::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:39,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:39,198::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:39,198::__init__::232::IOProcess::(_processLogs) (4376) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:39,199::__init__::232::IOProcess::(_processLogs) (4376) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:39,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:39,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:39,200::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:39,200::__init__::232::IOProcess::(_processLogs) (4377) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:39,201::__init__::232::IOProcess::(_processLogs) (4377) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:39,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:39,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:39,202::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:39,203::__init__::232::IOProcess::(_processLogs) (4378) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:39,203::__init__::232::IOProcess::(_processLogs) (4378) Queuing response >Dummy-33::DEBUG::2014-08-11 14:06:40,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:40,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135278 s, 75.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:41,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:41,302::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:41,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:42,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:42,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131104 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:44,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:44,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157042 s, 65.2 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:46,379::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:06:46,380::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9e9cf40-0218-480e-a08b-f3858a14384d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:46,381::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:46,388::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:46,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:46,391::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:06:46,394::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:46,394::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9e9cf40-0218-480e-a08b-f3858a14384d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:46,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9e9cf40-0218-480e-a08b-f3858a14384d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:46,395::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:46,395::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:46,395::task::993::Storage.TaskManager.Task::(_decref) Task=`d9e9cf40-0218-480e-a08b-f3858a14384d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:06:46,405::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:06:46,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`97f91f6f-05dc-4da3-b5cd-99b0b1246961`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:06:46,406::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:06:46,407::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6a0c6fc0-9a1f-4013-8dc1-0edac453cbef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:06:46,407::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:06:46,408::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:06:46,408::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6a0c6fc0-9a1f-4013-8dc1-0edac453cbef`::Granted request >Thread-34::DEBUG::2014-08-11 14:06:46,408::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`97f91f6f-05dc-4da3-b5cd-99b0b1246961`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:06:46,409::task::993::Storage.TaskManager.Task::(_decref) Task=`97f91f6f-05dc-4da3-b5cd-99b0b1246961`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:06:46,428::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:46,429::task::1191::Storage.TaskManager.Task::(prepare) Task=`97f91f6f-05dc-4da3-b5cd-99b0b1246961`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086505984', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:46,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`97f91f6f-05dc-4da3-b5cd-99b0b1246961`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:06:46,429::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:06:46,431::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:06:46,434::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:06:46,436::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:06:46,437::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:06:46,439::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:06:46,441::task::993::Storage.TaskManager.Task::(_decref) Task=`97f91f6f-05dc-4da3-b5cd-99b0b1246961`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:06:46,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:46,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131282 s, 78.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:06:48,370::task::595::Storage.TaskManager.Task::(_updateState) Task=`29fb27c6-0667-41d5-932d-9cdd35270269`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:48,371::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:06:48,371::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000500847', 'lastCheck': '9.2', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:48,372::task::1191::Storage.TaskManager.Task::(prepare) Task=`29fb27c6-0667-41d5-932d-9cdd35270269`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000500847', 'lastCheck': '9.2', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:06:48,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`29fb27c6-0667-41d5-932d-9cdd35270269`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:48,372::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:48,372::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:48,372::task::993::Storage.TaskManager.Task::(_decref) Task=`29fb27c6-0667-41d5-932d-9cdd35270269`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:06:48,393::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:48,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:48,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138861 s, 73.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:49,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:49,205::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:49,205::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:49,205::__init__::232::IOProcess::(_processLogs) (4379) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:49,205::__init__::232::IOProcess::(_processLogs) (4379) Queuing response >Thread-16::DEBUG::2014-08-11 14:06:49,206::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:06:49,224::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413204 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:49,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:49,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:49,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:49,226::__init__::232::IOProcess::(_processLogs) (4380) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:49,226::__init__::232::IOProcess::(_processLogs) (4380) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:49,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:49,228::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:49,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:49,229::__init__::232::IOProcess::(_processLogs) (4381) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:49,229::__init__::232::IOProcess::(_processLogs) (4381) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:49,229::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:49,230::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:49,231::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:49,231::__init__::232::IOProcess::(_processLogs) (4382) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:49,231::__init__::232::IOProcess::(_processLogs) (4382) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:49,231::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:49,232::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:49,232::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:49,233::__init__::232::IOProcess::(_processLogs) (4383) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:49,234::__init__::232::IOProcess::(_processLogs) (4383) Queuing response >Dummy-33::DEBUG::2014-08-11 14:06:50,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:50,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013356 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:51,398::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:51,399::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:51,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:06:52,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:52,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013215 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:54,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:55,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131806 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:56,420::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:56,426::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:06:56,459::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:06:56,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d15e8b1-ad26-46cb-b1ee-14c0a540e73d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:06:56,486::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:06:56,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:06:56,500::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:56,501::task::1191::Storage.TaskManager.Task::(prepare) Task=`0d15e8b1-ad26-46cb-b1ee-14c0a540e73d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:06:56,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d15e8b1-ad26-46cb-b1ee-14c0a540e73d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:06:56,501::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:06:56,502::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:06:56,502::task::993::Storage.TaskManager.Task::(_decref) Task=`0d15e8b1-ad26-46cb-b1ee-14c0a540e73d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:06:56,512::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:06:56,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c95d02e-b4fd-4fb7-b167-00095d1c05f8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:06:56,513::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:06:56,514::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`03830cbe-ae92-4009-a52a-886fb5eebeb7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:06:56,514::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:06:56,514::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:06:56,515::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`03830cbe-ae92-4009-a52a-886fb5eebeb7`::Granted request >Thread-34::DEBUG::2014-08-11 14:06:56,515::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1c95d02e-b4fd-4fb7-b167-00095d1c05f8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:06:56,515::task::993::Storage.TaskManager.Task::(_decref) Task=`1c95d02e-b4fd-4fb7-b167-00095d1c05f8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:06:56,536::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:56,538::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c95d02e-b4fd-4fb7-b167-00095d1c05f8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:06:56,539::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c95d02e-b4fd-4fb7-b167-00095d1c05f8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:06:56,540::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:06:56,541::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:06:56,543::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:06:56,545::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:06:56,546::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:06:56,547::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:06:56,549::task::993::Storage.TaskManager.Task::(_decref) Task=`1c95d02e-b4fd-4fb7-b167-00095d1c05f8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:06:57,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:57,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132628 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:06:59,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:06:59,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132865 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:59,234::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:59,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:59,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:59,235::__init__::232::IOProcess::(_processLogs) (4384) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:59,236::__init__::232::IOProcess::(_processLogs) (4384) Queuing response >Thread-16::DEBUG::2014-08-11 14:06:59,236::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:06:59,254::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427742 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:06:59,256::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:59,256::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:59,256::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:59,257::__init__::232::IOProcess::(_processLogs) (4385) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:06:59,257::__init__::232::IOProcess::(_processLogs) (4385) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:59,258::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:59,258::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:59,259::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:59,259::__init__::232::IOProcess::(_processLogs) (4386) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:59,259::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4386) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:59,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:59,261::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:59,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:59,262::__init__::232::IOProcess::(_processLogs) (4387) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:59,262::__init__::232::IOProcess::(_processLogs) (4387) Queuing response >Thread-14::DEBUG::2014-08-11 14:06:59,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:06:59,263::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:06:59,263::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:06:59,263::__init__::232::IOProcess::(_processLogs) (4388) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:06:59,263::__init__::232::IOProcess::(_processLogs) (4388) Queuing response >Dummy-33::DEBUG::2014-08-11 14:07:01,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:01,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133441 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:01,500::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:01,503::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:01,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:03,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:03,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132001 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:07:03,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`c732eedd-ed25-437e-843b-562e1ec45fcf`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:03,571::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:07:03,572::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427742', 'lastCheck': '4.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:03,573::task::1191::Storage.TaskManager.Task::(prepare) Task=`c732eedd-ed25-437e-843b-562e1ec45fcf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427742', 'lastCheck': '4.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:03,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`c732eedd-ed25-437e-843b-562e1ec45fcf`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:03,575::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:03,576::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:03,578::task::993::Storage.TaskManager.Task::(_decref) Task=`c732eedd-ed25-437e-843b-562e1ec45fcf`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:07:03,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:05,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:05,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131948 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:07:06,567::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:07:06,570::task::595::Storage.TaskManager.Task::(_updateState) Task=`3bf232ce-0c4c-4a30-8bb5-5dfbc906ed7d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:06,583::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:06,604::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:06,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:06,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:07:06,610::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:06,610::task::1191::Storage.TaskManager.Task::(prepare) Task=`3bf232ce-0c4c-4a30-8bb5-5dfbc906ed7d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:06,610::task::595::Storage.TaskManager.Task::(_updateState) Task=`3bf232ce-0c4c-4a30-8bb5-5dfbc906ed7d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:06,611::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:06,611::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:06,611::task::993::Storage.TaskManager.Task::(_decref) Task=`3bf232ce-0c4c-4a30-8bb5-5dfbc906ed7d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:07:06,623::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:07:06,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`78450a6a-82f8-412c-b866-5579215c756e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:07:06,624::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:07:06,626::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`754f9691-2ee3-4c28-84ef-a6c7d284a90b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:07:06,633::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:07:06,635::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:07:06,636::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`754f9691-2ee3-4c28-84ef-a6c7d284a90b`::Granted request >Thread-34::DEBUG::2014-08-11 14:07:06,638::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`78450a6a-82f8-412c-b866-5579215c756e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:07:06,639::task::993::Storage.TaskManager.Task::(_decref) Task=`78450a6a-82f8-412c-b866-5579215c756e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:07:06,698::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:06,699::task::1191::Storage.TaskManager.Task::(prepare) Task=`78450a6a-82f8-412c-b866-5579215c756e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:06,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`78450a6a-82f8-412c-b866-5579215c756e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:07:06,702::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:07:06,703::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:07:06,705::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:07:06,708::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:07:06,709::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:07:06,710::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:07:06,712::task::993::Storage.TaskManager.Task::(_decref) Task=`78450a6a-82f8-412c-b866-5579215c756e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:07:07,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:07,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133516 s, 76.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:07:07,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0c872d9-5846-4c0d-84c4-463f90c28524`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:07:07,595::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:07:07,596::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:07,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:07,597::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:07,597::__init__::232::IOProcess::(_processLogs) (4389) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:07:07,598::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4389) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:07,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:07,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:07:07,600::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:07:07,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:07:07,602::task::1191::Storage.TaskManager.Task::(prepare) Task=`f0c872d9-5846-4c0d-84c4-463f90c28524`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:07:07,602::__init__::232::IOProcess::(_processLogs) (4390) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:07:07,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0c872d9-5846-4c0d-84c4-463f90c28524`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:07:07,603::__init__::232::IOProcess::(_processLogs) (4390) Queuing response >Thread-55::DEBUG::2014-08-11 14:07:07,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:07:07,605::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:07:07,607::task::993::Storage.TaskManager.Task::(_decref) Task=`f0c872d9-5846-4c0d-84c4-463f90c28524`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:07:09,265::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:09,265::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:09,266::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:09,266::__init__::232::IOProcess::(_processLogs) (4391) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:09,267::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4391) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:09,268::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:07:09,286::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000410292 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:09,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:09,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:09,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:09,288::__init__::232::IOProcess::(_processLogs) (4392) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:09,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4392) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:09,289::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:09,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:09,290::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:09,290::__init__::232::IOProcess::(_processLogs) (4393) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:09,290::__init__::232::IOProcess::(_processLogs) (4393) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:09,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:09,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:09,292::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:09,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4394) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:09,292::__init__::232::IOProcess::(_processLogs) (4394) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:09,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:09,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:09,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:09,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4395) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:09,294::__init__::232::IOProcess::(_processLogs) (4395) Queuing response >Dummy-33::DEBUG::2014-08-11 14:07:09,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:09,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131804 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:11,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:11,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132508 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:11,615::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:11,616::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:11,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:13,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:13,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131955 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:15,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:15,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131462 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:16,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:16,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:07:16,728::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:07:16,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee6bf47a-e61c-40df-88d4-da785c3b49f1`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:16,788::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:16,790::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:07:16,804::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:16,806::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee6bf47a-e61c-40df-88d4-da785c3b49f1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:16,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee6bf47a-e61c-40df-88d4-da785c3b49f1`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:16,809::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:16,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:16,811::task::993::Storage.TaskManager.Task::(_decref) Task=`ee6bf47a-e61c-40df-88d4-da785c3b49f1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:07:16,825::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:07:16,828::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a073f57-758f-4a32-ba46-c33d5147b93f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:07:16,830::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:07:16,834::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ae49cf19-e7af-4141-baf7-6443d22b3e4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:07:16,835::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:07:16,838::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:07:16,839::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ae49cf19-e7af-4141-baf7-6443d22b3e4a`::Granted request >Thread-34::DEBUG::2014-08-11 14:07:16,844::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6a073f57-758f-4a32-ba46-c33d5147b93f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:07:16,845::task::993::Storage.TaskManager.Task::(_decref) Task=`6a073f57-758f-4a32-ba46-c33d5147b93f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:07:16,906::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:16,908::task::1191::Storage.TaskManager.Task::(prepare) Task=`6a073f57-758f-4a32-ba46-c33d5147b93f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:16,909::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a073f57-758f-4a32-ba46-c33d5147b93f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:07:16,910::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:07:16,911::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:07:16,914::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:07:16,917::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:07:16,918::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:07:16,919::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:07:16,922::task::993::Storage.TaskManager.Task::(_decref) Task=`6a073f57-758f-4a32-ba46-c33d5147b93f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:07:17,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:17,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131553 s, 77.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:07:18,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ee6892e-4df2-4fd9-a4e6-1539bf0fa5fc`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:18,714::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:07:18,714::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410292', 'lastCheck': '9.4', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:18,715::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ee6892e-4df2-4fd9-a4e6-1539bf0fa5fc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410292', 'lastCheck': '9.4', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:18,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ee6892e-4df2-4fd9-a4e6-1539bf0fa5fc`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:18,715::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:18,715::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:18,715::task::993::Storage.TaskManager.Task::(_decref) Task=`9ee6892e-4df2-4fd9-a4e6-1539bf0fa5fc`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:07:18,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:07:19,295::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:07:19,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,298::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,298::__init__::232::IOProcess::(_processLogs) (4396) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:07:19,298::__init__::232::IOProcess::(_processLogs) (4396) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:19,306::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:07:19,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,307::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,307::__init__::232::IOProcess::(_processLogs) (4397) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 14:07:19,308::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4397) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:19,308::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:07:19,309::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,309::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,310::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,310::__init__::232::IOProcess::(_processLogs) (4398) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:07:19,311::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4398) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:19,311::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:07:19,312::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,313::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,313::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,313::__init__::232::IOProcess::(_processLogs) (4399) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:19,313::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4399) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:19,314::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,314::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,315::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,315::__init__::232::IOProcess::(_processLogs) (4400) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:07:19,315::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4400) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:19,316::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 14:07:19,316::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:07:19,316::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:07:19,317::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,317::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,318::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,318::__init__::232::IOProcess::(_processLogs) (4401) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:19,318::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4401) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:19,319::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:07:19,336::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420654 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:19,337::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,338::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,338::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,338::__init__::232::IOProcess::(_processLogs) (4402) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:19,339::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4402) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:19,340::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,340::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,340::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,341::__init__::232::IOProcess::(_processLogs) (4403) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:19,341::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4403) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:19,342::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,342::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,342::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,343::__init__::232::IOProcess::(_processLogs) (4404) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:19,343::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4404) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:19,344::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:19,344::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:19,344::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:19,344::__init__::232::IOProcess::(_processLogs) (4405) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:19,345::__init__::232::IOProcess::(_processLogs) (4405) Queuing response >Dummy-33::DEBUG::2014-08-11 14:07:19,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:19,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132308 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:21,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:21,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133617 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:21,802::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:21,808::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:21,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:23,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:23,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131389 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:25,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:25,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01321 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:26,835::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:26,896::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:26,898::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:07:26,939::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:07:26,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d605442-96ea-4e4c-bb98-214bacaba8c4`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:26,940::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:07:26,957::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:26,958::task::1191::Storage.TaskManager.Task::(prepare) Task=`0d605442-96ea-4e4c-bb98-214bacaba8c4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:26,959::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d605442-96ea-4e4c-bb98-214bacaba8c4`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:26,960::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:26,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:26,963::task::993::Storage.TaskManager.Task::(_decref) Task=`0d605442-96ea-4e4c-bb98-214bacaba8c4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:07:26,979::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:07:26,982::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd652733-d786-4474-b734-792cac273a6d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:07:26,984::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:07:26,987::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b2222fa9-4e7d-43cd-b55a-fdc3913d5fbd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:07:26,988::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:07:26,990::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:07:26,991::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b2222fa9-4e7d-43cd-b55a-fdc3913d5fbd`::Granted request >Thread-34::DEBUG::2014-08-11 14:07:26,994::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dd652733-d786-4474-b734-792cac273a6d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:07:26,995::task::993::Storage.TaskManager.Task::(_decref) Task=`dd652733-d786-4474-b734-792cac273a6d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:07:27,052::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:27,054::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd652733-d786-4474-b734-792cac273a6d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:27,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd652733-d786-4474-b734-792cac273a6d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:07:27,056::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:07:27,057::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:07:27,059::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:07:27,060::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:07:27,061::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:07:27,063::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:07:27,065::task::993::Storage.TaskManager.Task::(_decref) Task=`dd652733-d786-4474-b734-792cac273a6d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:07:27,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:27,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132814 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:29,347::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:29,348::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:07:29,349::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:07:29,349::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:29,366::__init__::232::IOProcess::(_processLogs) (4406) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:29,367::__init__::232::IOProcess::(_processLogs) (4406) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:29,368::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000913706 s, 498 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:29,369::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:29,369::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:29,370::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:29,370::__init__::232::IOProcess::(_processLogs) (4407) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:29,371::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4407) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:29,372::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:29,373::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:29,373::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:29,373::__init__::232::IOProcess::(_processLogs) (4408) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:29,374::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4408) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:29,375::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:29,375::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:29,375::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:29,375::__init__::232::IOProcess::(_processLogs) (4409) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:29,376::__init__::232::IOProcess::(_processLogs) (4409) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:29,376::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:29,377::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:29,377::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:29,378::__init__::232::IOProcess::(_processLogs) (4410) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:29,379::__init__::232::IOProcess::(_processLogs) (4410) Queuing response >Dummy-33::DEBUG::2014-08-11 14:07:29,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:30,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134988 s, 75.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:31,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:31,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:31,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:32,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:32,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0163124 s, 62.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:07:33,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`c58223ef-74ca-4ea0-b255-4c373497df36`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:33,843::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:07:33,843::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000913706', 'lastCheck': '4.5', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:33,843::task::1191::Storage.TaskManager.Task::(prepare) Task=`c58223ef-74ca-4ea0-b255-4c373497df36`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000913706', 'lastCheck': '4.5', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:33,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`c58223ef-74ca-4ea0-b255-4c373497df36`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:33,844::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:33,844::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:33,844::task::993::Storage.TaskManager.Task::(_decref) Task=`c58223ef-74ca-4ea0-b255-4c373497df36`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:07:33,864::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:34,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:34,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131434 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:36,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:36,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131662 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:37,002::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:37,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:37,006::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:07:37,084::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:07:37,085::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01794f7-84ff-4be3-81fe-2900e379eea2`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:37,086::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:07:37,099::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:37,100::task::1191::Storage.TaskManager.Task::(prepare) Task=`a01794f7-84ff-4be3-81fe-2900e379eea2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:37,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`a01794f7-84ff-4be3-81fe-2900e379eea2`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:37,100::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:37,100::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:37,100::task::993::Storage.TaskManager.Task::(_decref) Task=`a01794f7-84ff-4be3-81fe-2900e379eea2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:07:37,111::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:07:37,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0272475-dda1-4a59-a8ad-a01cd15b608e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:07:37,112::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:07:37,113::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0d581142-8b65-4931-86c7-4f4b96e64f73`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:07:37,113::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:07:37,113::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:07:37,114::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0d581142-8b65-4931-86c7-4f4b96e64f73`::Granted request >Thread-34::DEBUG::2014-08-11 14:07:37,114::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e0272475-dda1-4a59-a8ad-a01cd15b608e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:07:37,114::task::993::Storage.TaskManager.Task::(_decref) Task=`e0272475-dda1-4a59-a8ad-a01cd15b608e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:07:37,169::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:37,171::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0272475-dda1-4a59-a8ad-a01cd15b608e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:37,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0272475-dda1-4a59-a8ad-a01cd15b608e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:07:37,173::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:07:37,175::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:07:37,177::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:07:37,178::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:07:37,179::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:07:37,180::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:07:37,182::task::993::Storage.TaskManager.Task::(_decref) Task=`e0272475-dda1-4a59-a8ad-a01cd15b608e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:07:38,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:38,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132948 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:39,380::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:39,380::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:39,380::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:39,381::__init__::232::IOProcess::(_processLogs) (4411) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:39,381::__init__::232::IOProcess::(_processLogs) (4411) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:39,382::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:07:39,399::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419233 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:39,400::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:39,401::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:39,401::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:39,401::__init__::232::IOProcess::(_processLogs) (4412) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:39,402::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4412) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:39,403::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:39,404::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:39,404::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:39,404::__init__::232::IOProcess::(_processLogs) (4413) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:39,405::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4413) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:39,405::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:39,406::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:39,406::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:39,406::__init__::232::IOProcess::(_processLogs) (4414) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:39,407::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4414) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:39,408::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:39,408::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:39,408::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:39,409::__init__::232::IOProcess::(_processLogs) (4415) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:39,409::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4415) Queuing response >Dummy-33::DEBUG::2014-08-11 14:07:40,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:40,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136474 s, 75.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:42,013::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:42,014::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:42,015::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:42,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:42,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138734 s, 73.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:44,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:44,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132642 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:46,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:46,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131879 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:47,019::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:47,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:47,022::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:07:47,198::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:07:47,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fd71ade-a808-49ca-9e1f-685f49d0817d`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:47,199::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:07:47,212::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:47,213::task::1191::Storage.TaskManager.Task::(prepare) Task=`6fd71ade-a808-49ca-9e1f-685f49d0817d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:47,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fd71ade-a808-49ca-9e1f-685f49d0817d`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:47,213::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:47,213::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:47,213::task::993::Storage.TaskManager.Task::(_decref) Task=`6fd71ade-a808-49ca-9e1f-685f49d0817d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:07:47,224::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:07:47,225::task::595::Storage.TaskManager.Task::(_updateState) Task=`abd7b6f4-a58b-41e9-9526-6afb80e9831b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:07:47,225::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:07:47,226::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b41b3310-f34e-4a07-a67e-41f251b4dc52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:07:47,226::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:07:47,226::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:07:47,227::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b41b3310-f34e-4a07-a67e-41f251b4dc52`::Granted request >Thread-34::DEBUG::2014-08-11 14:07:47,227::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`abd7b6f4-a58b-41e9-9526-6afb80e9831b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:07:47,228::task::993::Storage.TaskManager.Task::(_decref) Task=`abd7b6f4-a58b-41e9-9526-6afb80e9831b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:07:47,247::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:47,248::task::1191::Storage.TaskManager.Task::(prepare) Task=`abd7b6f4-a58b-41e9-9526-6afb80e9831b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:47,248::task::595::Storage.TaskManager.Task::(_updateState) Task=`abd7b6f4-a58b-41e9-9526-6afb80e9831b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:07:47,248::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:07:47,249::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:07:47,251::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:07:47,253::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:07:47,254::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:07:47,255::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:07:47,257::task::993::Storage.TaskManager.Task::(_decref) Task=`abd7b6f4-a58b-41e9-9526-6afb80e9831b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:07:48,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:48,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135715 s, 75.5 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:07:48,960::task::595::Storage.TaskManager.Task::(_updateState) Task=`e840a7be-9c25-4cb5-b4d1-34dc77e68822`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:48,960::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:07:48,961::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419233', 'lastCheck': '9.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:48,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`e840a7be-9c25-4cb5-b4d1-34dc77e68822`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419233', 'lastCheck': '9.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:07:48,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`e840a7be-9c25-4cb5-b4d1-34dc77e68822`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:48,961::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:48,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:48,962::task::993::Storage.TaskManager.Task::(_decref) Task=`e840a7be-9c25-4cb5-b4d1-34dc77e68822`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:07:48,982::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:07:49,411::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:49,411::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:49,412::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:49,412::__init__::232::IOProcess::(_processLogs) (4416) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:49,413::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4416) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:49,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:07:49,432::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426351 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:49,433::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:49,433::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:49,434::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:49,434::__init__::232::IOProcess::(_processLogs) (4417) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:49,434::__init__::232::IOProcess::(_processLogs) DEBUG|(4417) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:49,435::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:49,436::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:49,436::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:49,437::__init__::232::IOProcess::(_processLogs) (4418) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:49,437::__init__::232::IOProcess::(_processLogs) (4418) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:49,438::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:49,438::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:49,439::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:49,439::__init__::232::IOProcess::(_processLogs) (4419) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:49,439::__init__::232::IOProcess::(_processLogs) (4419) Queuing response >Thread-14::DEBUG::2014-08-11 14:07:49,439::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:49,440::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:49,441::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:49,441::__init__::232::IOProcess::(_processLogs) (4420) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:49,442::__init__::232::IOProcess::(_processLogs) (4420) Queuing response >Dummy-33::DEBUG::2014-08-11 14:07:50,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:50,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131625 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:52,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:52,030::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:52,030::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:07:52,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:52,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131097 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:54,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:54,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131085 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:07:56,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:56,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131566 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:57,038::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:57,039::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:07:57,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:07:57,274::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:07:57,277::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfdc885d-d6af-4951-a19a-df2b2d55590b`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:07:57,277::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:07:57,306::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:57,307::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfdc885d-d6af-4951-a19a-df2b2d55590b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:07:57,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfdc885d-d6af-4951-a19a-df2b2d55590b`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:07:57,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:07:57,308::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:07:57,309::task::993::Storage.TaskManager.Task::(_decref) Task=`bfdc885d-d6af-4951-a19a-df2b2d55590b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:07:57,320::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:07:57,321::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2171775-e6cf-4462-a000-76b166c52a87`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:07:57,321::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:07:57,322::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`24bf430d-5ebc-4068-9c62-9d18bb7ff212`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:07:57,322::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:07:57,323::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:07:57,323::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`24bf430d-5ebc-4068-9c62-9d18bb7ff212`::Granted request >Thread-34::DEBUG::2014-08-11 14:07:57,324::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c2171775-e6cf-4462-a000-76b166c52a87`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:07:57,324::task::993::Storage.TaskManager.Task::(_decref) Task=`c2171775-e6cf-4462-a000-76b166c52a87`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:07:57,349::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:57,350::task::1191::Storage.TaskManager.Task::(prepare) Task=`c2171775-e6cf-4462-a000-76b166c52a87`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:07:57,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2171775-e6cf-4462-a000-76b166c52a87`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:07:57,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:07:57,354::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:07:57,356::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:07:57,360::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:07:57,361::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:07:57,362::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:07:57,364::task::993::Storage.TaskManager.Task::(_decref) Task=`c2171775-e6cf-4462-a000-76b166c52a87`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:07:58,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:07:59,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013198 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:59,442::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:59,443::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:59,443::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:59,443::__init__::232::IOProcess::(_processLogs) (4421) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:59,444::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4421) Queuing response >Thread-16::DEBUG::2014-08-11 14:07:59,445::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:07:59,463::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423145 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:07:59,465::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:07:59,465::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:07:59,465::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:59,466::__init__::232::IOProcess::(_processLogs) (4422) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:07:59,467::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:59,467::__init__::232::IOProcess::(_processLogs) (4423) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:59,468::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:59,468::__init__::232::IOProcess::(_processLogs) (4424) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:07:59,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:07:59,469::__init__::232::IOProcess::(_processLogs) (4425) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:08:01,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:01,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132794 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:02,047::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:02,049::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:02,050::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:03,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:03,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013249 s, 77.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:08:04,073::task::595::Storage.TaskManager.Task::(_updateState) Task=`894a8ce8-9044-4561-86fd-4c804be912be`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:04,074::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:08:04,074::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423145', 'lastCheck': '4.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:04,074::task::1191::Storage.TaskManager.Task::(prepare) Task=`894a8ce8-9044-4561-86fd-4c804be912be`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423145', 'lastCheck': '4.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:04,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`894a8ce8-9044-4561-86fd-4c804be912be`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:04,075::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:04,075::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:04,075::task::993::Storage.TaskManager.Task::(_decref) Task=`894a8ce8-9044-4561-86fd-4c804be912be`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:08:04,093::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:05,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:05,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131874 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:07,053::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:07,054::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:07,055::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:07,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:07,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133361 s, 76.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:08:07,380::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:08:07,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`d035e3d5-42c1-4ccf-9e50-01f816a52648`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:07,382::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:08:07,394::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:07,395::task::1191::Storage.TaskManager.Task::(prepare) Task=`d035e3d5-42c1-4ccf-9e50-01f816a52648`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:07,395::task::595::Storage.TaskManager.Task::(_updateState) Task=`d035e3d5-42c1-4ccf-9e50-01f816a52648`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:07,395::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:07,395::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:07,396::task::993::Storage.TaskManager.Task::(_decref) Task=`d035e3d5-42c1-4ccf-9e50-01f816a52648`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:08:07,407::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:08:07,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f076a22-3ff0-4c06-a6df-97ddb8b1c886`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:08:07,408::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:08:07,409::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`528f19bf-8be0-4618-8043-3c1578744dfb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:08:07,409::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:08:07,409::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:08:07,410::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`528f19bf-8be0-4618-8043-3c1578744dfb`::Granted request >Thread-34::DEBUG::2014-08-11 14:08:07,410::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3f076a22-3ff0-4c06-a6df-97ddb8b1c886`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:08:07,410::task::993::Storage.TaskManager.Task::(_decref) Task=`3f076a22-3ff0-4c06-a6df-97ddb8b1c886`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:08:07,470::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:07,474::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f076a22-3ff0-4c06-a6df-97ddb8b1c886`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:07,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f076a22-3ff0-4c06-a6df-97ddb8b1c886`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:08:07,478::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:08:07,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:08:07,497::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:08:07,498::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:08:07,498::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:08:07,498::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:08:07,499::task::993::Storage.TaskManager.Task::(_decref) Task=`3f076a22-3ff0-4c06-a6df-97ddb8b1c886`::ref 0 aborting False >Thread-55::DEBUG::2014-08-11 14:08:07,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`e66437db-16ac-40a8-8b78-4db16f3f5643`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:08:07,717::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:08:07,718::__init__::232::IOProcess::(_processLogs) (4426) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:08:07,720::__init__::232::IOProcess::(_processLogs) (4427) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:08:07,720::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:08:07,720::task::1191::Storage.TaskManager.Task::(prepare) Task=`e66437db-16ac-40a8-8b78-4db16f3f5643`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:08:07,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`e66437db-16ac-40a8-8b78-4db16f3f5643`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:08:07,721::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:08:07,721::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:08:07,721::task::993::Storage.TaskManager.Task::(_decref) Task=`e66437db-16ac-40a8-8b78-4db16f3f5643`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:08:09,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:09,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131964 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:09,472::__init__::232::IOProcess::(_processLogs) (4428) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:08:09,473::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:08:09,490::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000440008 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:09,492::__init__::232::IOProcess::(_processLogs) (4429) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:08:09,493::__init__::232::IOProcess::(_processLogs) (4430) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:09,494::__init__::232::IOProcess::(_processLogs) (4431) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:09,495::__init__::232::IOProcess::(_processLogs) (4432) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:08:11,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:11,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132212 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:12,062::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:12,063::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:12,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:13,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:13,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131103 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:15,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:15,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132402 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:17,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:17,072::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:17,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:17,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-81::DEBUG::2014-08-11 14:08:17,516::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Dummy-33::DEBUG::2014-08-11 14:08:17,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134479 s, 76.1 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:08:17,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`2de1eb40-1c9f-4da4-873d-9636fe961717`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:17,522::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:08:17,542::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:17,543::task::1191::Storage.TaskManager.Task::(prepare) Task=`2de1eb40-1c9f-4da4-873d-9636fe961717`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:17,544::task::595::Storage.TaskManager.Task::(_updateState) Task=`2de1eb40-1c9f-4da4-873d-9636fe961717`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:17,545::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:17,546::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:17,547::task::993::Storage.TaskManager.Task::(_decref) Task=`2de1eb40-1c9f-4da4-873d-9636fe961717`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:08:17,559::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:08:17,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b689920-98ca-41ce-a628-c5aab9bb845d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:08:17,560::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:08:17,561::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`196cdbd4-44ae-43f9-94e4-fc3a31f5218a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:08:17,561::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:08:17,562::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:08:17,562::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`196cdbd4-44ae-43f9-94e4-fc3a31f5218a`::Granted request >Thread-34::DEBUG::2014-08-11 14:08:17,564::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8b689920-98ca-41ce-a628-c5aab9bb845d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:08:17,565::task::993::Storage.TaskManager.Task::(_decref) Task=`8b689920-98ca-41ce-a628-c5aab9bb845d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:08:17,660::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:17,662::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b689920-98ca-41ce-a628-c5aab9bb845d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:17,663::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b689920-98ca-41ce-a628-c5aab9bb845d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:08:17,664::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:08:17,665::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:08:17,667::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:08:17,669::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:08:17,670::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:08:17,671::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:08:17,673::task::993::Storage.TaskManager.Task::(_decref) Task=`8b689920-98ca-41ce-a628-c5aab9bb845d`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:08:19,184::task::595::Storage.TaskManager.Task::(_updateState) Task=`e637642e-881c-492b-b900-b1d1f0e6cf95`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:19,185::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:08:19,185::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000440008', 'lastCheck': '9.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:19,185::task::1191::Storage.TaskManager.Task::(prepare) Task=`e637642e-881c-492b-b900-b1d1f0e6cf95`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000440008', 'lastCheck': '9.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:19,186::task::595::Storage.TaskManager.Task::(_updateState) Task=`e637642e-881c-492b-b900-b1d1f0e6cf95`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:19,186::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:19,186::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:19,186::task::993::Storage.TaskManager.Task::(_decref) Task=`e637642e-881c-492b-b900-b1d1f0e6cf95`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:08:19,210::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:08:19,497::__init__::232::IOProcess::(_processLogs) (4433) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:08:19,498::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:08:19,516::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420722 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:19,517::__init__::232::IOProcess::(_processLogs) (4434) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:08:19,518::__init__::232::IOProcess::(_processLogs) (4435) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:19,519::__init__::232::IOProcess::(_processLogs) (4436) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:19,520::__init__::232::IOProcess::(_processLogs) (4437) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:08:19,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:19,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132408 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:21,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:21,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134393 s, 76.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:22,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:22,093::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:22,187::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:23,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:23,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130827 s, 78.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:25,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:25,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131772 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:27,194::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:27,195::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:27,287::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:08:27,690::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:08:27,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ad9531e-1eb6-47c5-9b2b-3f2d333a2981`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:27,691::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:08:27,710::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:27,710::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ad9531e-1eb6-47c5-9b2b-3f2d333a2981`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:27,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ad9531e-1eb6-47c5-9b2b-3f2d333a2981`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:27,711::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:27,711::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:27,711::task::993::Storage.TaskManager.Task::(_decref) Task=`4ad9531e-1eb6-47c5-9b2b-3f2d333a2981`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:08:27,733::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:08:27,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`be1365e4-ca27-43bd-9088-5c1756a05b32`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:08:27,742::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:08:27,751::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ba97ef13-5c40-43e1-8610-c6056ca70c24`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:08:27,751::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:08:27,752::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:08:27,755::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ba97ef13-5c40-43e1-8610-c6056ca70c24`::Granted request >Thread-34::DEBUG::2014-08-11 14:08:27,756::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`be1365e4-ca27-43bd-9088-5c1756a05b32`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:08:27,756::task::993::Storage.TaskManager.Task::(_decref) Task=`be1365e4-ca27-43bd-9088-5c1756a05b32`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:08:27,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 14:08:27,832::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:27,833::task::1191::Storage.TaskManager.Task::(prepare) Task=`be1365e4-ca27-43bd-9088-5c1756a05b32`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:27,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`be1365e4-ca27-43bd-9088-5c1756a05b32`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:08:27,835::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:08:27,837::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:08:27,840::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:08:27,842::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Dummy-33::DEBUG::2014-08-11 14:08:27,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269854 s, 37.9 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 14:08:27,844::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:08:27,845::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:08:27,848::task::993::Storage.TaskManager.Task::(_decref) Task=`be1365e4-ca27-43bd-9088-5c1756a05b32`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:08:29,522::__init__::232::IOProcess::(_processLogs) (4438) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:08:29,523::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:08:29,541::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424165 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:29,542::__init__::232::IOProcess::(_processLogs) (4439) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:08:29,544::__init__::232::IOProcess::(_processLogs) (4440) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:29,545::__init__::232::IOProcess::(_processLogs) (4441) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:29,546::__init__::232::IOProcess::(_processLogs) (4442) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:08:30,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:30,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131636 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:32,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:32,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0281648 s, 36.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:32,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:32,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:32,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:08:34,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`17110d51-e6b5-4302-a09a-8cf96df5cbdb`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:34,323::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:08:34,323::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424165', 'lastCheck': '4.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:34,323::task::1191::Storage.TaskManager.Task::(prepare) Task=`17110d51-e6b5-4302-a09a-8cf96df5cbdb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424165', 'lastCheck': '4.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:34,323::task::595::Storage.TaskManager.Task::(_updateState) Task=`17110d51-e6b5-4302-a09a-8cf96df5cbdb`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:34,324::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:34,324::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:34,324::task::993::Storage.TaskManager.Task::(_decref) Task=`17110d51-e6b5-4302-a09a-8cf96df5cbdb`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:08:34,342::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:34,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:34,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133721 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:36,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:36,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136295 s, 75.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:37,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:37,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:37,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:08:37,876::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:08:37,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e3eb0b0-19bd-4d5a-a21b-920a2297ad83`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:37,877::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:08:37,891::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:37,892::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e3eb0b0-19bd-4d5a-a21b-920a2297ad83`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:37,893::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e3eb0b0-19bd-4d5a-a21b-920a2297ad83`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:37,904::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:37,905::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:37,905::task::993::Storage.TaskManager.Task::(_decref) Task=`5e3eb0b0-19bd-4d5a-a21b-920a2297ad83`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:08:37,920::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:08:37,921::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c09a27b-1e63-4c83-bf0f-6bd119a14fb5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:08:37,921::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:08:37,922::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d16bbd2-6459-4335-9c31-285478c9db94`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:08:37,922::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:08:37,923::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:08:37,923::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d16bbd2-6459-4335-9c31-285478c9db94`::Granted request >Thread-34::DEBUG::2014-08-11 14:08:37,923::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2c09a27b-1e63-4c83-bf0f-6bd119a14fb5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:08:37,924::task::993::Storage.TaskManager.Task::(_decref) Task=`2c09a27b-1e63-4c83-bf0f-6bd119a14fb5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:08:37,960::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:37,960::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c09a27b-1e63-4c83-bf0f-6bd119a14fb5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:37,960::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c09a27b-1e63-4c83-bf0f-6bd119a14fb5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:08:37,961::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:08:37,961::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:08:37,961::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:08:37,962::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:08:37,962::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:08:37,962::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:08:37,963::task::993::Storage.TaskManager.Task::(_decref) Task=`2c09a27b-1e63-4c83-bf0f-6bd119a14fb5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:08:38,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:38,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132141 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:39,548::__init__::232::IOProcess::(_processLogs) (4443) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:08:39,549::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:08:39,567::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425867 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:39,568::__init__::232::IOProcess::(_processLogs) (4444) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:08:39,569::__init__::232::IOProcess::(_processLogs) (4445) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:39,570::__init__::232::IOProcess::(_processLogs) (4446) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:39,571::__init__::232::IOProcess::(_processLogs) (4447) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:08:40,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:40,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130991 s, 78.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:42,494::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:42,495::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:42,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:42,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:42,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132069 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:44,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:44,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133931 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:46,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:46,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132751 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:47,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:47,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:47,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:08:47,977::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:08:47,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9d8fb12-ee5f-4623-b4a8-885ef74db5b0`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:47,979::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:08:47,991::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:47,992::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9d8fb12-ee5f-4623-b4a8-885ef74db5b0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:47,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9d8fb12-ee5f-4623-b4a8-885ef74db5b0`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:47,992::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:47,992::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:47,993::task::993::Storage.TaskManager.Task::(_decref) Task=`b9d8fb12-ee5f-4623-b4a8-885ef74db5b0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:08:48,004::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:08:48,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`aad0eac2-00c0-4494-894b-13c2cd23cc3e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:08:48,005::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:08:48,005::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d9b2fff9-fdef-4e7f-b473-81389ac9c560`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:08:48,006::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:08:48,006::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:08:48,006::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d9b2fff9-fdef-4e7f-b473-81389ac9c560`::Granted request >Thread-34::DEBUG::2014-08-11 14:08:48,007::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aad0eac2-00c0-4494-894b-13c2cd23cc3e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:08:48,007::task::993::Storage.TaskManager.Task::(_decref) Task=`aad0eac2-00c0-4494-894b-13c2cd23cc3e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:08:48,027::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:48,027::task::1191::Storage.TaskManager.Task::(prepare) Task=`aad0eac2-00c0-4494-894b-13c2cd23cc3e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:48,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`aad0eac2-00c0-4494-894b-13c2cd23cc3e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:08:48,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:08:48,028::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:08:48,028::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:08:48,029::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:08:48,029::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:08:48,029::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:08:48,029::task::993::Storage.TaskManager.Task::(_decref) Task=`aad0eac2-00c0-4494-894b-13c2cd23cc3e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:08:48,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:48,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131494 s, 77.9 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:08:49,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`b49d15b0-c369-49d4-902a-6605125703e8`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:49,445::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:08:49,446::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425867', 'lastCheck': '9.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:49,446::task::1191::Storage.TaskManager.Task::(prepare) Task=`b49d15b0-c369-49d4-902a-6605125703e8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425867', 'lastCheck': '9.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:08:49,446::task::595::Storage.TaskManager.Task::(_updateState) Task=`b49d15b0-c369-49d4-902a-6605125703e8`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:49,447::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:49,447::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:49,447::task::993::Storage.TaskManager.Task::(_decref) Task=`b49d15b0-c369-49d4-902a-6605125703e8`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:08:49,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:08:49,573::__init__::232::IOProcess::(_processLogs) (4448) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:08:49,574::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:08:49,592::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0004154 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:49,593::__init__::232::IOProcess::(_processLogs) (4449) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:08:49,595::__init__::232::IOProcess::(_processLogs) (4450) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:49,596::__init__::232::IOProcess::(_processLogs) (4451) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:49,597::__init__::232::IOProcess::(_processLogs) (4452) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:08:50,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:50,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132269 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:52,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:52,787::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:52,788::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:08:52,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:52,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131041 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:54,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:55,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132079 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:08:57,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:57,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131316 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:57,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:57,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:08:57,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:08:58,041::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:08:58,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`be5a6d08-7171-42bd-8ae9-96fbcd39addd`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:08:58,043::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:08:58,055::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:58,056::task::1191::Storage.TaskManager.Task::(prepare) Task=`be5a6d08-7171-42bd-8ae9-96fbcd39addd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:08:58,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`be5a6d08-7171-42bd-8ae9-96fbcd39addd`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:08:58,056::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:08:58,056::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:08:58,057::task::993::Storage.TaskManager.Task::(_decref) Task=`be5a6d08-7171-42bd-8ae9-96fbcd39addd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:08:58,066::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:08:58,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`f83477fd-a033-4ea3-ab79-31059dcac5eb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:08:58,067::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:08:58,068::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fb69c8b6-6703-4c3c-bbf8-9d7a72247d85`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:08:58,068::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:08:58,069::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:08:58,069::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fb69c8b6-6703-4c3c-bbf8-9d7a72247d85`::Granted request >Thread-34::DEBUG::2014-08-11 14:08:58,070::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f83477fd-a033-4ea3-ab79-31059dcac5eb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:08:58,070::task::993::Storage.TaskManager.Task::(_decref) Task=`f83477fd-a033-4ea3-ab79-31059dcac5eb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:08:58,090::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:58,090::task::1191::Storage.TaskManager.Task::(prepare) Task=`f83477fd-a033-4ea3-ab79-31059dcac5eb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:08:58,090::task::595::Storage.TaskManager.Task::(_updateState) Task=`f83477fd-a033-4ea3-ab79-31059dcac5eb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:08:58,090::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:08:58,091::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:08:58,091::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:08:58,091::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:08:58,092::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:08:58,092::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:08:58,092::task::993::Storage.TaskManager.Task::(_decref) Task=`f83477fd-a033-4ea3-ab79-31059dcac5eb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:08:59,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:08:59,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131029 s, 78.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:59,599::__init__::232::IOProcess::(_processLogs) (4453) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:08:59,600::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:08:59,619::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413488 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:08:59,621::__init__::232::IOProcess::(_processLogs) (4454) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:08:59,622::__init__::232::IOProcess::(_processLogs) (4455) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:59,623::__init__::232::IOProcess::(_processLogs) (4456) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:08:59,624::__init__::232::IOProcess::(_processLogs) (4457) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:09:01,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:01,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133511 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:02,804::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:02,809::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:02,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:03,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:03,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132429 s, 77.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:09:04,566::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a1b698d-ad4b-4485-bd9f-75a4bbfb21e5`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:04,566::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:09:04,566::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413488', 'lastCheck': '4.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:09:04,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`0a1b698d-ad4b-4485-bd9f-75a4bbfb21e5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413488', 'lastCheck': '4.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:09:04,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a1b698d-ad4b-4485-bd9f-75a4bbfb21e5`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:04,567::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:04,567::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:04,568::task::993::Storage.TaskManager.Task::(_decref) Task=`0a1b698d-ad4b-4485-bd9f-75a4bbfb21e5`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:09:04,588::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:05,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:05,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136583 s, 75.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:07,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:07,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131967 s, 77.6 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:09:07,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`f981a133-7a25-4a2a-9e69-0eb88bab9e81`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:09:07,841::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:09:07,889::__init__::232::IOProcess::(_processLogs) (4458) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:09:07,891::__init__::232::IOProcess::(_processLogs) (4459) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:09:07,892::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:09:07,892::task::1191::Storage.TaskManager.Task::(prepare) Task=`f981a133-7a25-4a2a-9e69-0eb88bab9e81`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:09:07,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`f981a133-7a25-4a2a-9e69-0eb88bab9e81`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:09:07,892::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:09:07,893::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:09:07,893::task::993::Storage.TaskManager.Task::(_decref) Task=`f981a133-7a25-4a2a-9e69-0eb88bab9e81`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:07,903::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:07,904::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:07,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:09:08,105::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-81::DEBUG::2014-08-11 14:09:08,106::task::595::Storage.TaskManager.Task::(_updateState) Task=`60689de4-0cf7-4713-9efb-32a6fd259960`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:08,106::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:09:08,119::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:08,119::task::1191::Storage.TaskManager.Task::(prepare) Task=`60689de4-0cf7-4713-9efb-32a6fd259960`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:08,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`60689de4-0cf7-4713-9efb-32a6fd259960`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:08,120::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:08,120::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:08,120::task::993::Storage.TaskManager.Task::(_decref) Task=`60689de4-0cf7-4713-9efb-32a6fd259960`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:09:08,132::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 14:09:08,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`a95c5979-1141-419a-8f70-77cad04e3343`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:09:08,133::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:09:08,134::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8b704d97-0c56-4e79-8562-4bcb45a5ed41`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:09:08,134::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:09:08,134::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:09:08,134::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8b704d97-0c56-4e79-8562-4bcb45a5ed41`::Granted request >Thread-34::DEBUG::2014-08-11 14:09:08,135::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a95c5979-1141-419a-8f70-77cad04e3343`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:09:08,135::task::993::Storage.TaskManager.Task::(_decref) Task=`a95c5979-1141-419a-8f70-77cad04e3343`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:09:08,155::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:08,156::task::1191::Storage.TaskManager.Task::(prepare) Task=`a95c5979-1141-419a-8f70-77cad04e3343`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:08,156::task::595::Storage.TaskManager.Task::(_updateState) Task=`a95c5979-1141-419a-8f70-77cad04e3343`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:09:08,156::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:09:08,157::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:09:08,157::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:09:08,157::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:09:08,157::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:09:08,158::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:09:08,159::task::993::Storage.TaskManager.Task::(_decref) Task=`a95c5979-1141-419a-8f70-77cad04e3343`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:09:09,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:09,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131538 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:09,626::__init__::232::IOProcess::(_processLogs) (4460) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:09:09,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:09:09,645::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417141 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:09,646::__init__::232::IOProcess::(_processLogs) (4461) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:09:09,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:09:09,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:09,649::__init__::232::IOProcess::(_processLogs) (4462) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:09,650::__init__::232::IOProcess::(_processLogs) (4463) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:09,650::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:09,651::__init__::232::IOProcess::(_processLogs) (4464) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:09,651::__init__::232::IOProcess::(_processLogs) (4464) Queuing response >Dummy-33::DEBUG::2014-08-11 14:09:11,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:11,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013311 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:12,912::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:12,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:12,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:13,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:13,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131784 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:15,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:15,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131533 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:17,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:17,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134594 s, 76.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:17,996::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:17,998::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:18,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:09:18,173::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:09:18,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`62605e97-6b6c-4540-825c-e2d05a77cb33`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:18,174::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:09:18,187::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:18,187::task::1191::Storage.TaskManager.Task::(prepare) Task=`62605e97-6b6c-4540-825c-e2d05a77cb33`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:18,187::task::595::Storage.TaskManager.Task::(_updateState) Task=`62605e97-6b6c-4540-825c-e2d05a77cb33`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:18,188::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:18,188::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:18,188::task::993::Storage.TaskManager.Task::(_decref) Task=`62605e97-6b6c-4540-825c-e2d05a77cb33`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:09:18,198::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:09:18,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`a45444d4-c7b1-4e8a-97ba-482ea4a7bc61`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:09:18,199::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:09:18,200::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`558b2fe5-927e-4941-ae62-a304246a00eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:09:18,200::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:09:18,200::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:09:18,201::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`558b2fe5-927e-4941-ae62-a304246a00eb`::Granted request >Thread-34::DEBUG::2014-08-11 14:09:18,201::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a45444d4-c7b1-4e8a-97ba-482ea4a7bc61`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:09:18,201::task::993::Storage.TaskManager.Task::(_decref) Task=`a45444d4-c7b1-4e8a-97ba-482ea4a7bc61`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:09:18,241::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:18,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`a45444d4-c7b1-4e8a-97ba-482ea4a7bc61`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086440448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:18,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`a45444d4-c7b1-4e8a-97ba-482ea4a7bc61`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:09:18,241::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:09:18,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:09:18,242::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:09:18,242::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:09:18,243::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:09:18,243::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:09:18,243::task::993::Storage.TaskManager.Task::(_decref) Task=`a45444d4-c7b1-4e8a-97ba-482ea4a7bc61`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:09:19,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:09:19,668::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:19,669::__init__::232::IOProcess::(_processLogs) (4465) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:09:19,670::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-81::DEBUG::2014-08-11 14:09:19,695::task::595::Storage.TaskManager.Task::(_updateState) Task=`0075197c-6b1d-48c9-95a5-edd3a0e5113c`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:19,696::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-16::DEBUG::2014-08-11 14:09:19,697::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000590415 s, 771 kB/s\n'; <rc> = 0 >Thread-81::INFO::2014-08-11 14:09:19,698::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417141', 'lastCheck': '10.0', 'valid': True}} >Thread-14::DEBUG::2014-08-11 14:09:19,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-81::DEBUG::2014-08-11 14:09:19,703::task::1191::Storage.TaskManager.Task::(prepare) Task=`0075197c-6b1d-48c9-95a5-edd3a0e5113c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417141', 'lastCheck': '10.0', 'valid': True}} >Thread-14::DEBUG::2014-08-11 14:09:19,705::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:19,707::__init__::232::IOProcess::(_processLogs) (4466) Got request for method 'statvfs' >Thread-81::DEBUG::2014-08-11 14:09:19,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`0075197c-6b1d-48c9-95a5-edd3a0e5113c`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:19,709::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:19,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:19,712::task::993::Storage.TaskManager.Task::(_decref) Task=`0075197c-6b1d-48c9-95a5-edd3a0e5113c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:09:19,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0432142 s, 23.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:19,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:09:19,721::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:19,725::__init__::232::IOProcess::(_processLogs) (4467) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:19,729::__init__::232::IOProcess::(_processLogs) (4467) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:19,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:09:19,740::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:19,742::__init__::232::IOProcess::(_processLogs) (4468) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:19,743::__init__::232::IOProcess::(_processLogs) (4468) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:19,746::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:09:19,748::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:19,749::__init__::232::IOProcess::(_processLogs) (4469) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:19,751::__init__::232::IOProcess::(_processLogs) (4469) Queuing response >Thread-81::DEBUG::2014-08-11 14:09:19,759::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:21,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:21,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132441 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:23,093::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:23,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:23,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:23,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:23,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131605 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:25,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:25,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131995 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:27,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:27,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133013 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:28,193::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:28,194::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:09:28,255::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:09:28,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b2712d3-f3ea-42bd-8573-fcf3ca62aee8`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:28,293::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:28,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:09:28,307::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:28,308::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b2712d3-f3ea-42bd-8573-fcf3ca62aee8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:28,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b2712d3-f3ea-42bd-8573-fcf3ca62aee8`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:28,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:28,308::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:28,308::task::993::Storage.TaskManager.Task::(_decref) Task=`3b2712d3-f3ea-42bd-8573-fcf3ca62aee8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:09:28,320::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:09:28,321::task::595::Storage.TaskManager.Task::(_updateState) Task=`39945f45-a764-4ccc-96c8-e2464798a140`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:09:28,321::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:09:28,322::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`080a0818-7a3b-4b13-87bd-9d5ffb92927e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:09:28,322::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:09:28,322::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:09:28,322::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`080a0818-7a3b-4b13-87bd-9d5ffb92927e`::Granted request >Thread-34::DEBUG::2014-08-11 14:09:28,323::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`39945f45-a764-4ccc-96c8-e2464798a140`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:09:28,323::task::993::Storage.TaskManager.Task::(_decref) Task=`39945f45-a764-4ccc-96c8-e2464798a140`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:09:28,343::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:28,343::task::1191::Storage.TaskManager.Task::(prepare) Task=`39945f45-a764-4ccc-96c8-e2464798a140`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:28,344::task::595::Storage.TaskManager.Task::(_updateState) Task=`39945f45-a764-4ccc-96c8-e2464798a140`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:09:28,344::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:09:28,344::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:09:28,345::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:09:28,345::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:09:28,345::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:09:28,345::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:09:28,346::task::993::Storage.TaskManager.Task::(_decref) Task=`39945f45-a764-4ccc-96c8-e2464798a140`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:09:29,748::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:29,749::__init__::232::IOProcess::(_processLogs) (4470) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:09:29,749::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:09:29,767::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000496321 s, 917 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:29,768::__init__::232::IOProcess::(_processLogs) (4471) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:09:29,770::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:29,770::__init__::232::IOProcess::(_processLogs) (4472) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:29,771::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:29,771::__init__::232::IOProcess::(_processLogs) (4473) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:29,772::__init__::232::IOProcess::(_processLogs) (4473) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:29,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:29,773::__init__::232::IOProcess::(_processLogs) (4474) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:09:30,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:30,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133328 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:32,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:32,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233266 s, 43.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:33,302::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:33,303::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:33,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:34,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:34,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147841 s, 69.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:09:34,871::task::595::Storage.TaskManager.Task::(_updateState) Task=`4155eee1-7ae2-4828-af20-0ad06da6a290`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:34,872::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:09:34,872::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000496321', 'lastCheck': '5.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:09:34,872::task::1191::Storage.TaskManager.Task::(prepare) Task=`4155eee1-7ae2-4828-af20-0ad06da6a290`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000496321', 'lastCheck': '5.1', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:09:34,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`4155eee1-7ae2-4828-af20-0ad06da6a290`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:34,873::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:34,873::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:34,873::task::993::Storage.TaskManager.Task::(_decref) Task=`4155eee1-7ae2-4828-af20-0ad06da6a290`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:09:34,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:36,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:36,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132278 s, 77.4 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:09:38,359::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:09:38,360::task::595::Storage.TaskManager.Task::(_updateState) Task=`efed8f73-df1c-49ea-818d-3e69ac074349`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:38,389::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:38,397::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:38,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:38,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:38,421::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:09:38,418::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:38,423::task::1191::Storage.TaskManager.Task::(prepare) Task=`efed8f73-df1c-49ea-818d-3e69ac074349`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:38,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`efed8f73-df1c-49ea-818d-3e69ac074349`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:38,424::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:38,424::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:38,424::task::993::Storage.TaskManager.Task::(_decref) Task=`efed8f73-df1c-49ea-818d-3e69ac074349`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:09:38,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013782 s, 74.3 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 14:09:38,437::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:09:38,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00c8748-640a-401d-8995-55d76e1b61c5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:09:38,441::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:09:38,444::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0f97c93-be4a-4c6c-bc96-7fddc0e03ad6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:09:38,445::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:09:38,446::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:09:38,447::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0f97c93-be4a-4c6c-bc96-7fddc0e03ad6`::Granted request >Thread-34::DEBUG::2014-08-11 14:09:38,449::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f00c8748-640a-401d-8995-55d76e1b61c5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:09:38,450::task::993::Storage.TaskManager.Task::(_decref) Task=`f00c8748-640a-401d-8995-55d76e1b61c5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:09:38,495::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:38,497::task::1191::Storage.TaskManager.Task::(prepare) Task=`f00c8748-640a-401d-8995-55d76e1b61c5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:38,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00c8748-640a-401d-8995-55d76e1b61c5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:09:38,498::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:09:38,499::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:09:38,501::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:09:38,503::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:09:38,504::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:09:38,505::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:09:38,507::task::993::Storage.TaskManager.Task::(_decref) Task=`f00c8748-640a-401d-8995-55d76e1b61c5`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:09:39,775::__init__::232::IOProcess::(_processLogs) (4475) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:09:39,776::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:09:39,794::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00040939 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:39,795::__init__::232::IOProcess::(_processLogs) (4476) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:09:39,797::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:09:39,797::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:39,797::__init__::232::IOProcess::(_processLogs) (4477) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:39,798::__init__::232::IOProcess::(_processLogs) (4478) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:39,799::__init__::232::IOProcess::(_processLogs) (4478) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:39,799::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:39,800::__init__::232::IOProcess::(_processLogs) (4479) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:39,800::__init__::232::IOProcess::(_processLogs) (4479) Queuing response >Dummy-33::DEBUG::2014-08-11 14:09:40,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:40,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132779 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:42,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:42,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01328 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:43,426::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:43,428::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:43,490::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:44,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:44,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132598 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:46,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:46,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133084 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:48,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:48,499::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:09:48,523::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:09:48,592::task::595::Storage.TaskManager.Task::(_updateState) Task=`604d14a6-0de6-4ae3-bbd2-d9ef4b7bed11`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:48,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:09:48,594::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:09:48,608::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:48,609::task::1191::Storage.TaskManager.Task::(prepare) Task=`604d14a6-0de6-4ae3-bbd2-d9ef4b7bed11`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:48,609::task::595::Storage.TaskManager.Task::(_updateState) Task=`604d14a6-0de6-4ae3-bbd2-d9ef4b7bed11`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:48,609::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:48,609::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:48,610::task::993::Storage.TaskManager.Task::(_decref) Task=`604d14a6-0de6-4ae3-bbd2-d9ef4b7bed11`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:09:48,620::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:09:48,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad5044fa-1b07-47d4-a4ce-2393d9bbd69c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:09:48,621::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:09:48,621::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ea08e28a-61a2-4fd0-98bf-0e12377e3db1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:09:48,622::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:09:48,622::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:09:48,622::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ea08e28a-61a2-4fd0-98bf-0e12377e3db1`::Granted request >Thread-34::DEBUG::2014-08-11 14:09:48,623::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ad5044fa-1b07-47d4-a4ce-2393d9bbd69c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:09:48,623::task::993::Storage.TaskManager.Task::(_decref) Task=`ad5044fa-1b07-47d4-a4ce-2393d9bbd69c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:09:48,643::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:48,643::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad5044fa-1b07-47d4-a4ce-2393d9bbd69c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:48,643::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad5044fa-1b07-47d4-a4ce-2393d9bbd69c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:09:48,643::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:09:48,644::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:09:48,644::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:09:48,644::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:09:48,645::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:09:48,645::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:09:48,645::task::993::Storage.TaskManager.Task::(_decref) Task=`ad5044fa-1b07-47d4-a4ce-2393d9bbd69c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:09:48,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:48,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130995 s, 78.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:49,802::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:49,802::__init__::232::IOProcess::(_processLogs) (4480) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:09:49,803::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:09:49,821::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000518366 s, 878 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:49,823::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:49,824::__init__::232::IOProcess::(_processLogs) (4481) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:09:49,825::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:49,825::__init__::232::IOProcess::(_processLogs) (4482) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:49,826::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:49,826::__init__::232::IOProcess::(_processLogs) (4483) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:49,827::__init__::232::IOProcess::(_processLogs) (4483) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:49,828::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:49,828::__init__::232::IOProcess::(_processLogs) (4484) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:49,828::__init__::232::IOProcess::(_processLogs) (4484) Queuing response >Thread-81::DEBUG::2014-08-11 14:09:49,982::task::595::Storage.TaskManager.Task::(_updateState) Task=`660251d4-2678-4f73-8206-e95495afb8f7`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:49,982::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:09:49,983::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000518366', 'lastCheck': '0.2', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:09:49,983::task::1191::Storage.TaskManager.Task::(prepare) Task=`660251d4-2678-4f73-8206-e95495afb8f7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000518366', 'lastCheck': '0.2', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:09:49,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`660251d4-2678-4f73-8206-e95495afb8f7`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:49,983::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:49,984::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:49,984::task::993::Storage.TaskManager.Task::(_decref) Task=`660251d4-2678-4f73-8206-e95495afb8f7`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:09:50,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:50,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:50,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132517 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:52,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:52,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131821 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:53,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:53,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:53,690::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:09:54,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:54,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013151 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:09:56,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:56,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131784 s, 77.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:09:58,661::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:09:58,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c5fadb4-21d7-4747-8366-0dad3918765c`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:09:58,690::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:58,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:58,699::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:09:58,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:09:58,702::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:58,703::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c5fadb4-21d7-4747-8366-0dad3918765c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:09:58,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c5fadb4-21d7-4747-8366-0dad3918765c`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:09:58,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:09:58,703::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:09:58,704::task::993::Storage.TaskManager.Task::(_decref) Task=`4c5fadb4-21d7-4747-8366-0dad3918765c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:09:58,715::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:09:58,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b0ad4ca-f4f9-4e7b-ac03-d1178d2cb104`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:09:58,716::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:09:58,717::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9b5ba342-71ab-48f2-9d79-64d671e342cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:09:58,717::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:09:58,717::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:09:58,717::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9b5ba342-71ab-48f2-9d79-64d671e342cc`::Granted request >Thread-34::DEBUG::2014-08-11 14:09:58,720::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2b0ad4ca-f4f9-4e7b-ac03-d1178d2cb104`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:09:58,720::task::993::Storage.TaskManager.Task::(_decref) Task=`2b0ad4ca-f4f9-4e7b-ac03-d1178d2cb104`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:09:58,740::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:58,740::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b0ad4ca-f4f9-4e7b-ac03-d1178d2cb104`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:09:58,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b0ad4ca-f4f9-4e7b-ac03-d1178d2cb104`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:09:58,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:09:58,741::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:09:58,741::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:09:58,742::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:09:58,742::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:09:58,742::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:09:58,742::task::993::Storage.TaskManager.Task::(_decref) Task=`2b0ad4ca-f4f9-4e7b-ac03-d1178d2cb104`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:09:59,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:09:59,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218035 s, 47.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:59,830::__init__::232::IOProcess::(_processLogs) (4485) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:09:59,831::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:09:59,850::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418157 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:09:59,851::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:59,852::__init__::232::IOProcess::(_processLogs) (4486) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:09:59,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:09:59,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:59,854::__init__::232::IOProcess::(_processLogs) (4487) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:59,855::__init__::232::IOProcess::(_processLogs) (4488) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:09:59,855::__init__::232::IOProcess::(_processLogs) (4488) Queuing response >Thread-14::DEBUG::2014-08-11 14:09:59,856::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:09:59,856::__init__::232::IOProcess::(_processLogs) (4489) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:10:01,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:01,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015884 s, 64.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:03,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:03,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131397 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:03,707::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:03,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:03,709::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:10:05,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c51a17b-02fc-4bc5-b695-beeb1d84346b`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:05,110::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:10:05,110::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418157', 'lastCheck': '5.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:05,110::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c51a17b-02fc-4bc5-b695-beeb1d84346b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418157', 'lastCheck': '5.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:05,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c51a17b-02fc-4bc5-b695-beeb1d84346b`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:05,111::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:05,111::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:05,111::task::993::Storage.TaskManager.Task::(_decref) Task=`6c51a17b-02fc-4bc5-b695-beeb1d84346b`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:10:05,130::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:05,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:05,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013234 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:07,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:07,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132066 s, 77.5 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:10:08,011::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8bb047b-a286-4cf3-9109-6eee4c179611`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:10:08,011::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:10:08,012::__init__::232::IOProcess::(_processLogs) (4490) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:10:08,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:08,014::__init__::232::IOProcess::(_processLogs) (4491) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:10:08,015::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:10:08,015::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8bb047b-a286-4cf3-9109-6eee4c179611`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:10:08,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8bb047b-a286-4cf3-9109-6eee4c179611`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:10:08,015::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:10:08,016::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:10:08,016::task::993::Storage.TaskManager.Task::(_decref) Task=`e8bb047b-a286-4cf3-9109-6eee4c179611`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:08,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:08,717::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:08,718::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:10:08,755::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:10:08,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c399104-1481-4e0c-a65d-867b11cf88af`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:08,756::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:10:08,769::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:08,769::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c399104-1481-4e0c-a65d-867b11cf88af`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:08,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c399104-1481-4e0c-a65d-867b11cf88af`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:08,769::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:08,770::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:08,770::task::993::Storage.TaskManager.Task::(_decref) Task=`6c399104-1481-4e0c-a65d-867b11cf88af`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:10:08,780::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:10:08,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa016c93-e51a-4fcc-8419-52c977bd57e0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:10:08,781::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:10:08,782::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c10e01a3-c0d7-4357-81e7-7a723b08833f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:10:08,782::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:10:08,782::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:10:08,783::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c10e01a3-c0d7-4357-81e7-7a723b08833f`::Granted request >Thread-34::DEBUG::2014-08-11 14:10:08,783::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aa016c93-e51a-4fcc-8419-52c977bd57e0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:10:08,783::task::993::Storage.TaskManager.Task::(_decref) Task=`aa016c93-e51a-4fcc-8419-52c977bd57e0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:10:08,803::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:08,803::task::1191::Storage.TaskManager.Task::(prepare) Task=`aa016c93-e51a-4fcc-8419-52c977bd57e0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:08,804::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa016c93-e51a-4fcc-8419-52c977bd57e0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:10:08,804::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:10:08,804::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:10:08,805::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:10:08,805::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:10:08,805::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:10:08,805::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:10:08,806::task::993::Storage.TaskManager.Task::(_decref) Task=`aa016c93-e51a-4fcc-8419-52c977bd57e0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:10:09,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:09,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131709 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:09,859::__init__::232::IOProcess::(_processLogs) (4492) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:10:09,859::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:10:09,877::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042326 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:09,879::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:09,879::__init__::232::IOProcess::(_processLogs) (4493) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:10:09,880::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:09,880::__init__::232::IOProcess::(_processLogs) (4494) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:09,881::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:09,882::__init__::232::IOProcess::(_processLogs) (4495) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:09,882::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:09,883::__init__::232::IOProcess::(_processLogs) (4496) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:09,883::__init__::232::IOProcess::(_processLogs) (4496) Queuing response >Dummy-33::DEBUG::2014-08-11 14:10:11,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:11,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132039 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:13,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:13,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131521 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:13,725::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:13,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:13,727::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:15,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:15,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132741 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:17,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:17,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132403 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:18,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:18,735::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:18,790::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:10:18,825::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:10:18,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`a809e862-03c3-429c-9e41-5e8afc8d5f00`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:18,827::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:10:18,839::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:18,840::task::1191::Storage.TaskManager.Task::(prepare) Task=`a809e862-03c3-429c-9e41-5e8afc8d5f00`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:18,840::task::595::Storage.TaskManager.Task::(_updateState) Task=`a809e862-03c3-429c-9e41-5e8afc8d5f00`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:18,840::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:18,840::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:18,840::task::993::Storage.TaskManager.Task::(_decref) Task=`a809e862-03c3-429c-9e41-5e8afc8d5f00`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:10:18,871::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:10:18,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`03e1bfd6-97a3-4961-abc1-3f2ccdbf8877`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:10:18,872::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:10:18,873::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`566fab37-09d4-4140-a001-af5a427a5489`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:10:18,873::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:10:18,873::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:10:18,873::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`566fab37-09d4-4140-a001-af5a427a5489`::Granted request >Thread-34::DEBUG::2014-08-11 14:10:18,874::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`03e1bfd6-97a3-4961-abc1-3f2ccdbf8877`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:10:18,874::task::993::Storage.TaskManager.Task::(_decref) Task=`03e1bfd6-97a3-4961-abc1-3f2ccdbf8877`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:10:18,894::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:18,894::task::1191::Storage.TaskManager.Task::(prepare) Task=`03e1bfd6-97a3-4961-abc1-3f2ccdbf8877`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:18,895::task::595::Storage.TaskManager.Task::(_updateState) Task=`03e1bfd6-97a3-4961-abc1-3f2ccdbf8877`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:10:18,895::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:10:18,895::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:10:18,895::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:10:18,896::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:10:18,896::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:10:18,896::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:10:18,896::task::993::Storage.TaskManager.Task::(_decref) Task=`03e1bfd6-97a3-4961-abc1-3f2ccdbf8877`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:10:19,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:19,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133319 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:19,885::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:19,885::__init__::232::IOProcess::(_processLogs) (4497) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:10:19,886::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:10:19,903::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417774 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:19,904::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:19,905::__init__::232::IOProcess::(_processLogs) (4498) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:10:19,906::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:19,906::__init__::232::IOProcess::(_processLogs) (4499) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:19,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:19,907::__init__::232::IOProcess::(_processLogs) (4500) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:19,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:19,909::__init__::232::IOProcess::(_processLogs) (4501) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:19,909::__init__::232::IOProcess::(_processLogs) (4501) Queuing response >Thread-81::DEBUG::2014-08-11 14:10:20,219::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca4f4988-d9ef-4b7c-8724-ff013af8068f`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:20,219::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:10:20,220::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417774', 'lastCheck': '0.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:20,220::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca4f4988-d9ef-4b7c-8724-ff013af8068f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417774', 'lastCheck': '0.3', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:20,220::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca4f4988-d9ef-4b7c-8724-ff013af8068f`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:20,221::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:20,221::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:20,221::task::993::Storage.TaskManager.Task::(_decref) Task=`ca4f4988-d9ef-4b7c-8724-ff013af8068f`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:10:20,241::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:21,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:21,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133049 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:23,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:23,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131833 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:23,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:23,796::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:23,890::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:25,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:25,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131614 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:27,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:27,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132093 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:28,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:28,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:10:28,930::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:10:28,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`d09c4d9f-cd10-4c2b-8144-cc2fdcdd9c9c`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:28,994::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:10:28,995::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:10:29,009::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:29,010::task::1191::Storage.TaskManager.Task::(prepare) Task=`d09c4d9f-cd10-4c2b-8144-cc2fdcdd9c9c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:29,010::task::595::Storage.TaskManager.Task::(_updateState) Task=`d09c4d9f-cd10-4c2b-8144-cc2fdcdd9c9c`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:29,010::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:29,010::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:29,011::task::993::Storage.TaskManager.Task::(_decref) Task=`d09c4d9f-cd10-4c2b-8144-cc2fdcdd9c9c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:10:29,021::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:10:29,022::task::595::Storage.TaskManager.Task::(_updateState) Task=`b982c153-f207-4c2b-a9f3-9e5eb228d3ff`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:10:29,022::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:10:29,023::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`62888f55-67c0-4762-9b66-7cfdd0b4d244`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:10:29,023::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:10:29,023::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:10:29,024::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`62888f55-67c0-4762-9b66-7cfdd0b4d244`::Granted request >Thread-34::DEBUG::2014-08-11 14:10:29,024::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b982c153-f207-4c2b-a9f3-9e5eb228d3ff`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:10:29,024::task::993::Storage.TaskManager.Task::(_decref) Task=`b982c153-f207-4c2b-a9f3-9e5eb228d3ff`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:10:29,044::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:29,045::task::1191::Storage.TaskManager.Task::(prepare) Task=`b982c153-f207-4c2b-a9f3-9e5eb228d3ff`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:29,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`b982c153-f207-4c2b-a9f3-9e5eb228d3ff`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:10:29,045::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:10:29,045::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:10:29,046::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:10:29,046::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:10:29,046::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:10:29,046::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:10:29,047::task::993::Storage.TaskManager.Task::(_decref) Task=`b982c153-f207-4c2b-a9f3-9e5eb228d3ff`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:10:29,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:29,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132831 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:29,911::__init__::232::IOProcess::(_processLogs) (4502) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:10:29,912::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:10:29,929::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041353 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:29,930::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:29,930::__init__::232::IOProcess::(_processLogs) (4503) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:10:29,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:29,932::__init__::232::IOProcess::(_processLogs) (4504) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:29,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:29,933::__init__::232::IOProcess::(_processLogs) (4505) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:29,934::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:29,934::__init__::232::IOProcess::(_processLogs) (4506) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:29,934::__init__::232::IOProcess::(_processLogs) (4506) Queuing response >Dummy-33::DEBUG::2014-08-11 14:10:31,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:31,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278775 s, 36.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:34,003::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:34,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:34,090::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:34,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:34,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131717 s, 77.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:10:35,342::task::595::Storage.TaskManager.Task::(_updateState) Task=`c022515f-1c38-47ba-af85-55cce7379022`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:35,342::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:10:35,343::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041353', 'lastCheck': '5.4', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:35,343::task::1191::Storage.TaskManager.Task::(prepare) Task=`c022515f-1c38-47ba-af85-55cce7379022`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041353', 'lastCheck': '5.4', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:35,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`c022515f-1c38-47ba-af85-55cce7379022`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:35,343::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:35,344::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:35,344::task::993::Storage.TaskManager.Task::(_decref) Task=`c022515f-1c38-47ba-af85-55cce7379022`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:10:35,364::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:36,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:36,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0149161 s, 68.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:38,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:38,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132154 s, 77.5 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:10:39,059::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:10:39,060::task::595::Storage.TaskManager.Task::(_updateState) Task=`873b4d2e-fe7b-47b7-8dd2-c1806f5d4455`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:39,089::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:39,097::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:39,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:39,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:10:39,103::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:39,103::task::1191::Storage.TaskManager.Task::(prepare) Task=`873b4d2e-fe7b-47b7-8dd2-c1806f5d4455`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:39,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`873b4d2e-fe7b-47b7-8dd2-c1806f5d4455`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:39,104::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:39,104::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:39,104::task::993::Storage.TaskManager.Task::(_decref) Task=`873b4d2e-fe7b-47b7-8dd2-c1806f5d4455`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:10:39,116::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:10:39,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7ed0a40-360d-4c82-8fc4-ea53128902ca`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:10:39,117::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:10:39,118::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`13c61066-0fc1-4891-be89-f21980523cf2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:10:39,118::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:10:39,119::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:10:39,119::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`13c61066-0fc1-4891-be89-f21980523cf2`::Granted request >Thread-34::DEBUG::2014-08-11 14:10:39,119::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e7ed0a40-360d-4c82-8fc4-ea53128902ca`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:10:39,120::task::993::Storage.TaskManager.Task::(_decref) Task=`e7ed0a40-360d-4c82-8fc4-ea53128902ca`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:10:39,139::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:39,140::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7ed0a40-360d-4c82-8fc4-ea53128902ca`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:39,140::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7ed0a40-360d-4c82-8fc4-ea53128902ca`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:10:39,140::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:10:39,140::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:10:39,141::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:10:39,141::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:10:39,141::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:10:39,141::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:10:39,142::task::993::Storage.TaskManager.Task::(_decref) Task=`e7ed0a40-360d-4c82-8fc4-ea53128902ca`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:10:39,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:10:39,937::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:39,937::__init__::232::IOProcess::(_processLogs) (4507) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:10:39,938::__init__::232::IOProcess::(_processLogs) DEBUG|(4507) Queuing response >Thread-16::DEBUG::2014-08-11 14:10:39,939::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:10:39,956::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413972 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:39,958::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:10:39,958::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:10:39,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:39,958::__init__::232::IOProcess::(_processLogs) (4508) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:10:39,959::__init__::232::IOProcess::(_processLogs) DEBUG|(4508) Queuing response >Thread-14::DEBUG::2014-08-11 14:10:39,961::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:10:39,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:10:39,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:39,962::__init__::232::IOProcess::(_processLogs) (4509) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:39,963::__init__::232::IOProcess::(_processLogs) (4509) Queuing response >Thread-14::DEBUG::2014-08-11 14:10:39,963::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:39,963::__init__::232::IOProcess::(_processLogs) (4510) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:39,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:39,964::__init__::232::IOProcess::(_processLogs) (4511) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:10:40,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:40,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134615 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:42,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:42,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132228 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:44,106::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:44,107::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:44,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:44,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:44,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178534 s, 57.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:46,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:46,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132486 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:48,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:48,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132131 s, 77.5 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:10:49,154::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:10:49,155::task::595::Storage.TaskManager.Task::(_updateState) Task=`645ddb25-6cf1-4f97-9773-723525db4ece`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:49,190::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:49,197::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:49,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:49,200::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:10:49,202::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:49,203::task::1191::Storage.TaskManager.Task::(prepare) Task=`645ddb25-6cf1-4f97-9773-723525db4ece`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:49,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`645ddb25-6cf1-4f97-9773-723525db4ece`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:49,203::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:49,203::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:49,204::task::993::Storage.TaskManager.Task::(_decref) Task=`645ddb25-6cf1-4f97-9773-723525db4ece`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:10:49,215::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:10:49,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`867f3b2c-b91c-42d7-82cd-72b6023e2636`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:10:49,216::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:10:49,217::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3779a80d-4920-41f4-acce-23e6bd660edb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:10:49,217::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:10:49,217::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:10:49,218::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3779a80d-4920-41f4-acce-23e6bd660edb`::Granted request >Thread-34::DEBUG::2014-08-11 14:10:49,218::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`867f3b2c-b91c-42d7-82cd-72b6023e2636`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:10:49,219::task::993::Storage.TaskManager.Task::(_decref) Task=`867f3b2c-b91c-42d7-82cd-72b6023e2636`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:10:49,238::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:49,239::task::1191::Storage.TaskManager.Task::(prepare) Task=`867f3b2c-b91c-42d7-82cd-72b6023e2636`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086374912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:49,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`867f3b2c-b91c-42d7-82cd-72b6023e2636`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:10:49,239::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:10:49,239::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:10:49,240::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:10:49,240::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:10:49,240::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:10:49,240::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:10:49,241::task::993::Storage.TaskManager.Task::(_decref) Task=`867f3b2c-b91c-42d7-82cd-72b6023e2636`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:10:49,967::__init__::232::IOProcess::(_processLogs) (4512) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:10:49,968::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:10:49,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428353 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:10:49,987::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:49,987::__init__::232::IOProcess::(_processLogs) (4513) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:10:49,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:49,989::__init__::232::IOProcess::(_processLogs) (4514) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:49,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:49,990::__init__::232::IOProcess::(_processLogs) (4515) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:49,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:49,991::__init__::232::IOProcess::(_processLogs) (4516) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:10:49,991::__init__::232::IOProcess::(_processLogs) (4516) Queuing response >Thread-81::DEBUG::2014-08-11 14:10:50,462::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1fa19af-05aa-450e-a3e0-b306aef79560`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:50,463::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:10:50,463::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428353', 'lastCheck': '0.5', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:50,463::task::1191::Storage.TaskManager.Task::(prepare) Task=`b1fa19af-05aa-450e-a3e0-b306aef79560`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428353', 'lastCheck': '0.5', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:10:50,464::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1fa19af-05aa-450e-a3e0-b306aef79560`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:50,464::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:50,464::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:50,464::task::993::Storage.TaskManager.Task::(_decref) Task=`b1fa19af-05aa-450e-a3e0-b306aef79560`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:10:50,483::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:50,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:50,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018309 s, 55.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:52,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:52,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216203 s, 47.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:54,204::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:54,205::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:54,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:10:54,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:54,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132113 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:56,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:56,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132465 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:10:58,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:10:58,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136236 s, 75.2 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:10:59,253::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:10:59,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`333874ee-7b9a-45f1-960b-2df404668c4e`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:10:59,289::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:59,297::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:59,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:10:59,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:10:59,302::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:59,303::task::1191::Storage.TaskManager.Task::(prepare) Task=`333874ee-7b9a-45f1-960b-2df404668c4e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:10:59,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`333874ee-7b9a-45f1-960b-2df404668c4e`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:10:59,303::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:10:59,304::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:10:59,304::task::993::Storage.TaskManager.Task::(_decref) Task=`333874ee-7b9a-45f1-960b-2df404668c4e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:10:59,314::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:10:59,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`b890939c-95fd-462b-b134-783ff205a087`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:10:59,315::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:10:59,316::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e95204b-a59a-4953-b93b-26ba92412002`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:10:59,316::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:10:59,317::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:10:59,317::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e95204b-a59a-4953-b93b-26ba92412002`::Granted request >Thread-34::DEBUG::2014-08-11 14:10:59,317::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b890939c-95fd-462b-b134-783ff205a087`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:10:59,318::task::993::Storage.TaskManager.Task::(_decref) Task=`b890939c-95fd-462b-b134-783ff205a087`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:10:59,342::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:59,343::task::1191::Storage.TaskManager.Task::(prepare) Task=`b890939c-95fd-462b-b134-783ff205a087`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:10:59,344::task::595::Storage.TaskManager.Task::(_updateState) Task=`b890939c-95fd-462b-b134-783ff205a087`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:10:59,345::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:10:59,346::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:10:59,348::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:10:59,350::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:10:59,351::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:10:59,352::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:10:59,354::task::993::Storage.TaskManager.Task::(_decref) Task=`b890939c-95fd-462b-b134-783ff205a087`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:10:59,993::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:10:59,994::__init__::232::IOProcess::(_processLogs) (4517) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:10:59,995::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:11:00,013::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432245 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:11:00,014::__init__::232::IOProcess::(_processLogs) (4518) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:11:00,016::__init__::232::IOProcess::(_processLogs) (4519) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:00,017::__init__::232::IOProcess::(_processLogs) (4520) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:00,018::__init__::232::IOProcess::(_processLogs) (4521) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:11:00,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:00,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132352 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:02,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:02,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134176 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:04,307::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:04,308::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:04,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:04,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:05,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133314 s, 76.8 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:11:05,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3dd47f6-d64d-4832-a987-e8a5eac55397`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:05,575::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:11:05,575::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432245', 'lastCheck': '5.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:05,575::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3dd47f6-d64d-4832-a987-e8a5eac55397`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432245', 'lastCheck': '5.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:05,575::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3dd47f6-d64d-4832-a987-e8a5eac55397`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:05,576::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:05,576::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:05,576::task::993::Storage.TaskManager.Task::(_decref) Task=`f3dd47f6-d64d-4832-a987-e8a5eac55397`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:11:05,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:07,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:07,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132897 s, 77.1 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:11:08,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`c13aaec3-c7cc-4883-9f28-bab156da4a56`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:11:08,121::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:11:08,122::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:08,122::__init__::232::IOProcess::(_processLogs) (4522) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:11:08,124::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:08,124::__init__::232::IOProcess::(_processLogs) (4523) Queuing response >Thread-55::INFO::2014-08-11 14:11:08,125::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:11:08,125::task::1191::Storage.TaskManager.Task::(prepare) Task=`c13aaec3-c7cc-4883-9f28-bab156da4a56`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:11:08,125::task::595::Storage.TaskManager.Task::(_updateState) Task=`c13aaec3-c7cc-4883-9f28-bab156da4a56`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:11:08,125::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:11:08,125::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:11:08,126::task::993::Storage.TaskManager.Task::(_decref) Task=`c13aaec3-c7cc-4883-9f28-bab156da4a56`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:11:09,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:09,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132595 s, 77.2 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:11:09,369::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:11:09,370::task::595::Storage.TaskManager.Task::(_updateState) Task=`69702f6b-96b1-4392-888c-6664c5f3adce`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:09,389::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:09,398::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:09,399::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:09,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:11:09,402::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:09,402::task::1191::Storage.TaskManager.Task::(prepare) Task=`69702f6b-96b1-4392-888c-6664c5f3adce`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:09,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`69702f6b-96b1-4392-888c-6664c5f3adce`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:09,403::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:09,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:09,403::task::993::Storage.TaskManager.Task::(_decref) Task=`69702f6b-96b1-4392-888c-6664c5f3adce`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:11:09,414::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:11:09,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f67ca9f-63d0-4410-8b0a-21f5c12e4a8c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:11:09,415::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:11:09,415::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6689140f-1ae5-4a08-9084-eef88fa67d66`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:11:09,416::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:11:09,416::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:11:09,416::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6689140f-1ae5-4a08-9084-eef88fa67d66`::Granted request >Thread-34::DEBUG::2014-08-11 14:11:09,417::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8f67ca9f-63d0-4410-8b0a-21f5c12e4a8c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:11:09,417::task::993::Storage.TaskManager.Task::(_decref) Task=`8f67ca9f-63d0-4410-8b0a-21f5c12e4a8c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:11:09,437::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:09,439::task::1191::Storage.TaskManager.Task::(prepare) Task=`8f67ca9f-63d0-4410-8b0a-21f5c12e4a8c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:09,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`8f67ca9f-63d0-4410-8b0a-21f5c12e4a8c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:11:09,441::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:11:09,442::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:11:09,444::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:11:09,446::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:11:09,447::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:11:09,448::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:11:09,450::task::993::Storage.TaskManager.Task::(_decref) Task=`8f67ca9f-63d0-4410-8b0a-21f5c12e4a8c`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:11:10,023::__init__::232::IOProcess::(_processLogs) (4524) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:11:10,024::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:11:10,053::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00663971 s, 68.5 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:11:10,057::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:10,058::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:10,058::__init__::232::IOProcess::(_processLogs) (4525) Queuing response >Thread-14::DEBUG::2014-08-11 14:11:10,060::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:10,060::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:10,061::__init__::232::IOProcess::(_processLogs) (4526) Queuing response >Thread-14::DEBUG::2014-08-11 14:11:10,062::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:10,062::__init__::232::IOProcess::(_processLogs) (4527) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:10,063::__init__::232::IOProcess::(_processLogs) (4527) Queuing response >Thread-14::DEBUG::2014-08-11 14:11:10,064::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:10,064::__init__::232::IOProcess::(_processLogs) (4528) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:10,064::__init__::232::IOProcess::(_processLogs) (4528) Queuing response >Dummy-33::DEBUG::2014-08-11 14:11:11,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:11,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132256 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:13,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:13,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132021 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:14,407::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:14,409::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:14,491::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:15,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:15,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132713 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:17,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:17,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131484 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:19,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:19,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132125 s, 77.5 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:11:19,468::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-81::DEBUG::2014-08-11 14:11:19,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`aba2ca30-c4bd-45d6-b762-8bbb156d62ae`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:19,469::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:11:19,482::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:19,483::task::1191::Storage.TaskManager.Task::(prepare) Task=`aba2ca30-c4bd-45d6-b762-8bbb156d62ae`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:19,483::task::595::Storage.TaskManager.Task::(_updateState) Task=`aba2ca30-c4bd-45d6-b762-8bbb156d62ae`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:19,483::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:19,483::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:19,483::task::993::Storage.TaskManager.Task::(_decref) Task=`aba2ca30-c4bd-45d6-b762-8bbb156d62ae`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:11:19,495::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 14:11:19,496::task::595::Storage.TaskManager.Task::(_updateState) Task=`52001994-f67d-427d-9f91-c3dc3b6d2971`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:11:19,496::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:11:19,497::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2f51ce82-7f92-4a11-af23-a8fa5342aede`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:11:19,497::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:11:19,499::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:19,500::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:11:19,500::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2f51ce82-7f92-4a11-af23-a8fa5342aede`::Granted request >Thread-34::DEBUG::2014-08-11 14:11:19,504::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`52001994-f67d-427d-9f91-c3dc3b6d2971`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:11:19,505::task::993::Storage.TaskManager.Task::(_decref) Task=`52001994-f67d-427d-9f91-c3dc3b6d2971`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:19,506::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:19,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:11:19,537::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:19,539::task::1191::Storage.TaskManager.Task::(prepare) Task=`52001994-f67d-427d-9f91-c3dc3b6d2971`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:19,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`52001994-f67d-427d-9f91-c3dc3b6d2971`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:11:19,541::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:11:19,542::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:11:19,544::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:11:19,547::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:11:19,548::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:11:19,549::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:11:19,551::task::993::Storage.TaskManager.Task::(_decref) Task=`52001994-f67d-427d-9f91-c3dc3b6d2971`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:11:20,066::__init__::232::IOProcess::(_processLogs) (4529) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:11:20,067::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:11:20,085::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415243 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:11:20,087::__init__::232::IOProcess::(_processLogs) (4530) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:11:20,088::__init__::232::IOProcess::(_processLogs) (4531) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:20,089::__init__::232::IOProcess::(_processLogs) (4532) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:20,090::__init__::232::IOProcess::(_processLogs) (4533) Got request for method 'access' >Thread-81::DEBUG::2014-08-11 14:11:20,686::task::595::Storage.TaskManager.Task::(_updateState) Task=`66b7677c-91d0-47d4-9ceb-921e3f202da2`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:20,687::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:11:20,687::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415243', 'lastCheck': '0.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:20,687::task::1191::Storage.TaskManager.Task::(prepare) Task=`66b7677c-91d0-47d4-9ceb-921e3f202da2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415243', 'lastCheck': '0.6', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:20,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`66b7677c-91d0-47d4-9ceb-921e3f202da2`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:20,688::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:20,688::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:20,688::task::993::Storage.TaskManager.Task::(_decref) Task=`66b7677c-91d0-47d4-9ceb-921e3f202da2`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:11:20,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:21,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:21,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131771 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:23,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:23,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132038 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:24,521::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:24,522::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:24,591::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:25,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:25,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133972 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:27,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:27,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131119 s, 78.1 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:11:29,567::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:11:29,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`39199291-f2af-4ebe-8224-658c68973f3e`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:29,590::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:29,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:29,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:29,601::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:11:29,603::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:29,604::task::1191::Storage.TaskManager.Task::(prepare) Task=`39199291-f2af-4ebe-8224-658c68973f3e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:29,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`39199291-f2af-4ebe-8224-658c68973f3e`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:29,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:29,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:29,605::task::993::Storage.TaskManager.Task::(_decref) Task=`39199291-f2af-4ebe-8224-658c68973f3e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:11:29,615::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:11:29,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`0696beee-7d8b-4568-97d6-c3e0794bb69f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:11:29,616::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:11:29,617::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f8a7d48-5b63-4bbe-b348-d6d65fab7f06`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:11:29,617::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:11:29,618::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:11:29,618::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f8a7d48-5b63-4bbe-b348-d6d65fab7f06`::Granted request >Thread-34::DEBUG::2014-08-11 14:11:29,619::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0696beee-7d8b-4568-97d6-c3e0794bb69f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:11:29,619::task::993::Storage.TaskManager.Task::(_decref) Task=`0696beee-7d8b-4568-97d6-c3e0794bb69f`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:11:29,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 14:11:29,651::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:29,653::task::1191::Storage.TaskManager.Task::(prepare) Task=`0696beee-7d8b-4568-97d6-c3e0794bb69f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:29,653::task::595::Storage.TaskManager.Task::(_updateState) Task=`0696beee-7d8b-4568-97d6-c3e0794bb69f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:11:29,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:11:29,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:11:29,657::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:11:29,659::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:11:29,659::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:11:29,660::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:11:29,662::task::993::Storage.TaskManager.Task::(_decref) Task=`0696beee-7d8b-4568-97d6-c3e0794bb69f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:11:29,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324547 s, 31.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:11:30,092::__init__::232::IOProcess::(_processLogs) (4534) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:11:30,093::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:11:30,112::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417688 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:11:30,113::__init__::232::IOProcess::(_processLogs) (4535) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:11:30,115::__init__::232::IOProcess::(_processLogs) (4536) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:30,116::__init__::232::IOProcess::(_processLogs) (4537) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:30,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:30,117::__init__::232::IOProcess::(_processLogs) (4538) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:11:31,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:31,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132734 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:33,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:33,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133261 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:34,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:34,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:34,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:11:35,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`04d585a4-5334-4a2d-95e6-5209e59ab0e7`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:35,810::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:11:35,811::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417688', 'lastCheck': '5.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:35,811::task::1191::Storage.TaskManager.Task::(prepare) Task=`04d585a4-5334-4a2d-95e6-5209e59ab0e7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417688', 'lastCheck': '5.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:35,811::task::595::Storage.TaskManager.Task::(_updateState) Task=`04d585a4-5334-4a2d-95e6-5209e59ab0e7`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:35,811::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:35,812::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:35,812::task::993::Storage.TaskManager.Task::(_decref) Task=`04d585a4-5334-4a2d-95e6-5209e59ab0e7`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:11:35,830::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:35,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:36,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132611 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:38,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:38,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131287 s, 78.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:11:39,679::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:11:39,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3f92446-86e4-43ab-b02a-30a4b1b54067`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:39,691::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:39,699::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:39,701::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:39,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:11:39,705::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:39,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3f92446-86e4-43ab-b02a-30a4b1b54067`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:39,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3f92446-86e4-43ab-b02a-30a4b1b54067`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:39,706::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:39,706::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:39,706::task::993::Storage.TaskManager.Task::(_decref) Task=`e3f92446-86e4-43ab-b02a-30a4b1b54067`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:11:39,716::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:11:39,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1617f3c-26c1-4538-8916-2945492c9511`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:11:39,717::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:11:39,718::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`80f10a6c-af22-41ec-b0fe-8dd870d75ec2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:11:39,718::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:11:39,718::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:11:39,719::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`80f10a6c-af22-41ec-b0fe-8dd870d75ec2`::Granted request >Thread-34::DEBUG::2014-08-11 14:11:39,719::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d1617f3c-26c1-4538-8916-2945492c9511`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:11:39,719::task::993::Storage.TaskManager.Task::(_decref) Task=`d1617f3c-26c1-4538-8916-2945492c9511`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:11:39,739::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:39,739::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1617f3c-26c1-4538-8916-2945492c9511`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:39,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1617f3c-26c1-4538-8916-2945492c9511`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:11:39,740::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:11:39,740::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:11:39,740::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:11:39,741::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:11:39,741::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:11:39,741::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:11:39,742::task::993::Storage.TaskManager.Task::(_decref) Task=`d1617f3c-26c1-4538-8916-2945492c9511`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:11:40,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:11:40,120::__init__::232::IOProcess::(_processLogs) (4539) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:11:40,123::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:40,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0324565 s, 31.5 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:11:40,149::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000608542 s, 748 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:11:40,152::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:40,153::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:40,155::__init__::232::IOProcess::(_processLogs) (4540) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:11:40,158::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:40,160::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:40,161::__init__::232::IOProcess::(_processLogs) (4541) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:40,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:40,164::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:40,165::__init__::232::IOProcess::(_processLogs) (4542) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:40,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:40,168::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:40,169::__init__::232::IOProcess::(_processLogs) (4543) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:11:42,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:42,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141842 s, 72.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:44,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:44,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132337 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:44,709::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:44,710::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:44,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:46,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:46,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131987 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:48,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:48,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131999 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:11:49,754::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-81::DEBUG::2014-08-11 14:11:49,756::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e0cdb76-14ed-4f43-ab16-c2f69202ddbc`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:49,792::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:49,801::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:49,804::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:49,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:11:49,806::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:49,807::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e0cdb76-14ed-4f43-ab16-c2f69202ddbc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:49,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e0cdb76-14ed-4f43-ab16-c2f69202ddbc`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:49,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:49,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:49,810::task::993::Storage.TaskManager.Task::(_decref) Task=`4e0cdb76-14ed-4f43-ab16-c2f69202ddbc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:11:49,820::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-34::DEBUG::2014-08-11 14:11:49,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cf2778f-1aa8-40b9-b0a7-fa6f4315b1b4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:11:49,821::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:11:49,822::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`044d08b7-66a3-4253-96c9-84495d649992`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:11:49,822::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:11:49,823::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:11:49,823::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`044d08b7-66a3-4253-96c9-84495d649992`::Granted request >Thread-34::DEBUG::2014-08-11 14:11:49,824::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3cf2778f-1aa8-40b9-b0a7-fa6f4315b1b4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:11:49,824::task::993::Storage.TaskManager.Task::(_decref) Task=`3cf2778f-1aa8-40b9-b0a7-fa6f4315b1b4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:11:49,843::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:49,844::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cf2778f-1aa8-40b9-b0a7-fa6f4315b1b4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:49,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cf2778f-1aa8-40b9-b0a7-fa6f4315b1b4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:11:49,844::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:11:49,845::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:11:49,845::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:11:49,846::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:11:49,846::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:11:49,846::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:11:49,846::task::993::Storage.TaskManager.Task::(_decref) Task=`3cf2778f-1aa8-40b9-b0a7-fa6f4315b1b4`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:11:50,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:50,172::__init__::232::IOProcess::(_processLogs) (4544) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:11:50,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:11:50,191::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421536 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:11:50,192::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:50,193::__init__::232::IOProcess::(_processLogs) (4545) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:11:50,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:50,194::__init__::232::IOProcess::(_processLogs) (4546) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:50,195::__init__::232::IOProcess::(_processLogs) DEBUG|(4546) Queuing response >Thread-14::DEBUG::2014-08-11 14:11:50,195::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:11:50,196::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:50,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:50,196::__init__::232::IOProcess::(_processLogs) DEBUG|(4547) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:50,197::__init__::232::IOProcess::(_processLogs) (4547) Queuing response >Thread-14::DEBUG::2014-08-11 14:11:50,197::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:11:50,198::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:11:50,198::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:11:50,198::__init__::232::IOProcess::(_processLogs) (4548) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:11:50,199::__init__::232::IOProcess::(_processLogs) (4548) Queuing response >Dummy-33::DEBUG::2014-08-11 14:11:50,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:50,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132777 s, 77.1 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:11:50,934::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a4f542b-09d5-4129-879d-3c3e9dc59682`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:50,935::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:11:50,935::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421536', 'lastCheck': '0.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:50,936::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a4f542b-09d5-4129-879d-3c3e9dc59682`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421536', 'lastCheck': '0.7', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:11:50,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a4f542b-09d5-4129-879d-3c3e9dc59682`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:50,936::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:50,936::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:50,936::task::993::Storage.TaskManager.Task::(_decref) Task=`2a4f542b-09d5-4129-879d-3c3e9dc59682`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:11:50,960::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:52,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:52,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135533 s, 75.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:54,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:54,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135752 s, 75.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:54,811::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:54,812::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:54,813::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:11:56,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:56,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132931 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:11:58,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:11:58,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132257 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:59,818::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:59,820::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:11:59,858::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:11:59,896::task::595::Storage.TaskManager.Task::(_updateState) Task=`0582399a-cf7d-44d4-b760-1852acaa3719`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:11:59,897::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:11:59,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:11:59,911::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:59,911::task::1191::Storage.TaskManager.Task::(prepare) Task=`0582399a-cf7d-44d4-b760-1852acaa3719`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:11:59,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`0582399a-cf7d-44d4-b760-1852acaa3719`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:11:59,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:11:59,912::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:11:59,912::task::993::Storage.TaskManager.Task::(_decref) Task=`0582399a-cf7d-44d4-b760-1852acaa3719`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:11:59,923::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:11:59,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`07b12dca-0beb-491b-b725-d9f15def6c3e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:11:59,924::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:11:59,924::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`df3c3bdb-28fb-4a22-a3f4-47e2b1b187dc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:11:59,925::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:11:59,925::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:11:59,925::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`df3c3bdb-28fb-4a22-a3f4-47e2b1b187dc`::Granted request >Thread-34::DEBUG::2014-08-11 14:11:59,926::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`07b12dca-0beb-491b-b725-d9f15def6c3e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:11:59,926::task::993::Storage.TaskManager.Task::(_decref) Task=`07b12dca-0beb-491b-b725-d9f15def6c3e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:11:59,946::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:59,947::task::1191::Storage.TaskManager.Task::(prepare) Task=`07b12dca-0beb-491b-b725-d9f15def6c3e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:11:59,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`07b12dca-0beb-491b-b725-d9f15def6c3e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:11:59,947::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:11:59,947::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:11:59,948::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:11:59,948::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:11:59,948::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:11:59,948::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:11:59,949::task::993::Storage.TaskManager.Task::(_decref) Task=`07b12dca-0beb-491b-b725-d9f15def6c3e`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:12:00,201::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:00,201::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:00,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:00,202::__init__::232::IOProcess::(_processLogs) (4549) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:00,202::__init__::232::IOProcess::(_processLogs) DEBUG|(4549) Queuing response >Thread-16::DEBUG::2014-08-11 14:12:00,204::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:12:00,222::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414639 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:12:00,223::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:00,224::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:00,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:00,224::__init__::232::IOProcess::(_processLogs) (4550) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:00,225::__init__::232::IOProcess::(_processLogs) DEBUG|(4550) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:00,227::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:00,227::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:00,227::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:00,227::__init__::232::IOProcess::(_processLogs) (4551) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:00,228::__init__::232::IOProcess::(_processLogs) DEBUG|(4551) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:00,229::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:00,229::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:00,230::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:00,230::__init__::232::IOProcess::(_processLogs) (4552) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:00,230::__init__::232::IOProcess::(_processLogs) (4552) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:00,231::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:00,231::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:00,231::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:00,232::__init__::232::IOProcess::(_processLogs) (4553) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:00,232::__init__::232::IOProcess::(_processLogs) DEBUG|(4553) Queuing response >Dummy-33::DEBUG::2014-08-11 14:12:00,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:00,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174717 s, 58.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:02,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:02,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243065 s, 42.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:04,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:04,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:04,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:05,012::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:05,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0198107 s, 51.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:12:06,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`843e40bc-cc6a-4201-8341-4b2d5782f706`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:06,053::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:12:06,053::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414639', 'lastCheck': '5.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:06,054::task::1191::Storage.TaskManager.Task::(prepare) Task=`843e40bc-cc6a-4201-8341-4b2d5782f706`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414639', 'lastCheck': '5.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:06,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`843e40bc-cc6a-4201-8341-4b2d5782f706`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:06,054::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:06,054::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:06,054::task::993::Storage.TaskManager.Task::(_decref) Task=`843e40bc-cc6a-4201-8341-4b2d5782f706`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:12:06,073::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:07,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:07,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133287 s, 76.8 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:12:08,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`69a51ec4-f1e4-4140-b909-fe1f6a52b868`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:12:08,240::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:12:08,241::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:08,241::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:08,242::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:08,242::__init__::232::IOProcess::(_processLogs) (4554) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:12:08,242::__init__::232::IOProcess::(_processLogs) DEBUG|(4554) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:08,243::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:08,244::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:12:08,244::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:12:08,245::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:08,246::__init__::232::IOProcess::(_processLogs) (4555) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:12:08,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`69a51ec4-f1e4-4140-b909-fe1f6a52b868`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:12:08,247::__init__::232::IOProcess::(_processLogs) (4555) Queuing response >Thread-55::DEBUG::2014-08-11 14:12:08,248::task::595::Storage.TaskManager.Task::(_updateState) Task=`69a51ec4-f1e4-4140-b909-fe1f6a52b868`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:12:08,248::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:12:08,248::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:12:08,249::task::993::Storage.TaskManager.Task::(_decref) Task=`69a51ec4-f1e4-4140-b909-fe1f6a52b868`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:12:09,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:09,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134386 s, 76.2 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:12:09,961::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:12:09,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7379814-8b48-4164-b3fe-50d5b5b28e76`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:09,994::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-81::INFO::2014-08-11 14:12:10,007::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:10,008::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7379814-8b48-4164-b3fe-50d5b5b28e76`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:10,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7379814-8b48-4164-b3fe-50d5b5b28e76`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:10,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:10,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:10,008::task::993::Storage.TaskManager.Task::(_decref) Task=`d7379814-8b48-4164-b3fe-50d5b5b28e76`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:12:10,019::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:12:10,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`b10b6657-5cb3-4bc9-9c39-fa4c2e6b78b2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:12:10,020::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:10,023::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:12:10,024::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`180a8ef1-e2c9-4341-a0f2-7a258bf0c47e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:12:10,025::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:12:10,027::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:10,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:12:10,029::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`180a8ef1-e2c9-4341-a0f2-7a258bf0c47e`::Granted request >Thread-34::DEBUG::2014-08-11 14:12:10,032::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b10b6657-5cb3-4bc9-9c39-fa4c2e6b78b2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:10,034::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:12:10,034::task::993::Storage.TaskManager.Task::(_decref) Task=`b10b6657-5cb3-4bc9-9c39-fa4c2e6b78b2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:12:10,056::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:10,056::task::1191::Storage.TaskManager.Task::(prepare) Task=`b10b6657-5cb3-4bc9-9c39-fa4c2e6b78b2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086309376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:10,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`b10b6657-5cb3-4bc9-9c39-fa4c2e6b78b2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:12:10,057::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:12:10,057::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:12:10,057::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:12:10,058::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:12:10,058::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:12:10,058::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:12:10,058::task::993::Storage.TaskManager.Task::(_decref) Task=`b10b6657-5cb3-4bc9-9c39-fa4c2e6b78b2`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:12:10,234::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:10,234::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:10,235::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:10,236::__init__::232::IOProcess::(_processLogs) (4556) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:12:10,236::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:12:10,237::__init__::232::IOProcess::(_processLogs) (4556) Queuing response >Thread-16::DEBUG::2014-08-11 14:12:10,255::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416346 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:12:10,257::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:10,257::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:10,257::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:10,257::__init__::232::IOProcess::(_processLogs) (4557) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:10,258::__init__::232::IOProcess::(_processLogs) DEBUG|(4557) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:10,260::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:10,260::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:10,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:10,261::__init__::232::IOProcess::(_processLogs) (4558) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:10,261::__init__::232::IOProcess::(_processLogs) (4558) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:10,262::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:10,262::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:10,262::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:10,263::__init__::232::IOProcess::(_processLogs) (4559) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:10,263::__init__::232::IOProcess::(_processLogs) DEBUG|(4559) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:10,264::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:10,265::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:10,265::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:10,266::__init__::232::IOProcess::(_processLogs) (4560) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:10,267::__init__::232::IOProcess::(_processLogs) (4560) Queuing response >Dummy-33::DEBUG::2014-08-11 14:12:11,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:11,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013183 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:13,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:13,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131649 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:15,043::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:15,045::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:15,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:15,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:15,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132309 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:17,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:17,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132426 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:19,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:19,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133612 s, 76.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:12:20,070::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:12:20,071::task::595::Storage.TaskManager.Task::(_updateState) Task=`6eacfb2d-d6fc-4eda-b25b-be406f124d2f`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:20,094::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:20,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:20,105::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:20,106::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:12:20,107::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:20,107::task::1191::Storage.TaskManager.Task::(prepare) Task=`6eacfb2d-d6fc-4eda-b25b-be406f124d2f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:20,108::task::595::Storage.TaskManager.Task::(_updateState) Task=`6eacfb2d-d6fc-4eda-b25b-be406f124d2f`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:20,108::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:20,108::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:20,108::task::993::Storage.TaskManager.Task::(_decref) Task=`6eacfb2d-d6fc-4eda-b25b-be406f124d2f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:12:20,121::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:12:20,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`1422cb1e-0626-4523-85a8-877de3b966e7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:12:20,122::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:12:20,122::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e108d835-5e77-4b8d-bb63-952ecd54987f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:12:20,123::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:12:20,123::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:12:20,123::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e108d835-5e77-4b8d-bb63-952ecd54987f`::Granted request >Thread-34::DEBUG::2014-08-11 14:12:20,124::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1422cb1e-0626-4523-85a8-877de3b966e7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:12:20,124::task::993::Storage.TaskManager.Task::(_decref) Task=`1422cb1e-0626-4523-85a8-877de3b966e7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:12:20,144::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:20,144::task::1191::Storage.TaskManager.Task::(prepare) Task=`1422cb1e-0626-4523-85a8-877de3b966e7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:20,144::task::595::Storage.TaskManager.Task::(_updateState) Task=`1422cb1e-0626-4523-85a8-877de3b966e7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:12:20,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:12:20,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:12:20,145::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:12:20,145::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:12:20,146::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:12:20,146::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:12:20,146::task::993::Storage.TaskManager.Task::(_decref) Task=`1422cb1e-0626-4523-85a8-877de3b966e7`::ref 0 aborting False >Thread-16::DEBUG::2014-08-11 14:12:20,267::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:12:20,277::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,278::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,283::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,287::__init__::232::IOProcess::(_processLogs) (4561) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:12:20,290::__init__::232::IOProcess::(_processLogs) (4561) Queuing response >Thread-16::DEBUG::2014-08-11 14:12:20,313::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:12:20,315::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,316::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,318::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,320::__init__::232::IOProcess::(_processLogs) DEBUG|(4562) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 14:12:20,320::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:12:20,321::__init__::232::IOProcess::(_processLogs) (4562) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:20,324::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,325::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,326::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:12:20,329::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:12:20,330::__init__::232::IOProcess::(_processLogs) DEBUG|(4563) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:12:20,331::__init__::232::IOProcess::(_processLogs) (4563) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:20,337::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,340::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,341::__init__::232::IOProcess::(_processLogs) (4564) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:20,342::__init__::232::IOProcess::(_processLogs) (4564) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:20,345::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,347::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,348::__init__::232::IOProcess::(_processLogs) (4565) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:12:20,350::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 14:12:20,350::__init__::232::IOProcess::(_processLogs) DEBUG|(4565) Queuing response >Thread-16::INFO::2014-08-11 14:12:20,351::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:12:20,352::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:12:20,354::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,358::__init__::232::IOProcess::(_processLogs) (4566) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:20,360::__init__::232::IOProcess::(_processLogs) DEBUG|(4566) Queuing response >Thread-16::DEBUG::2014-08-11 14:12:20,362::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:12:20,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000489488 s, 930 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:12:20,385::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,387::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,388::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,389::__init__::232::IOProcess::(_processLogs) (4567) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:20,391::__init__::232::IOProcess::(_processLogs) DEBUG|(4567) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:20,395::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,396::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,397::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,398::__init__::232::IOProcess::(_processLogs) (4568) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:20,400::__init__::232::IOProcess::(_processLogs) DEBUG|(4568) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:20,403::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,405::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,407::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,409::__init__::232::IOProcess::(_processLogs) (4569) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:20,411::__init__::232::IOProcess::(_processLogs) (4569) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:20,414::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:20,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:20,418::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:20,420::__init__::232::IOProcess::(_processLogs) (4570) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:20,421::__init__::232::IOProcess::(_processLogs) (4570) Queuing response >Thread-81::DEBUG::2014-08-11 14:12:21,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c01c5ca-bee2-494e-8245-a9143a1e4089`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:21,178::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:12:21,178::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000489488', 'lastCheck': '0.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:21,178::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c01c5ca-bee2-494e-8245-a9143a1e4089`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000489488', 'lastCheck': '0.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:21,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c01c5ca-bee2-494e-8245-a9143a1e4089`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:21,179::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:21,179::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:21,179::task::993::Storage.TaskManager.Task::(_decref) Task=`8c01c5ca-bee2-494e-8245-a9143a1e4089`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:12:21,202::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:21,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:21,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013265 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:23,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:23,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132552 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:25,115::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:25,116::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:25,195::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:25,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:25,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131653 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:27,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:27,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131555 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:29,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:29,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131886 s, 77.6 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:12:30,161::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:12:30,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`811cb7fd-d80a-48bf-8add-d626723c92a7`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:30,195::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:30,203::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:30,206::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:12:30,208::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:30,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::DEBUG::2014-08-11 14:12:30,210::task::1191::Storage.TaskManager.Task::(prepare) Task=`811cb7fd-d80a-48bf-8add-d626723c92a7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:30,211::task::595::Storage.TaskManager.Task::(_updateState) Task=`811cb7fd-d80a-48bf-8add-d626723c92a7`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:30,213::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:30,213::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:30,213::task::993::Storage.TaskManager.Task::(_decref) Task=`811cb7fd-d80a-48bf-8add-d626723c92a7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:12:30,223::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:12:30,224::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4266372-8fc4-4b3d-b765-c37ff5efed25`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:12:30,224::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:12:30,225::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9d9c2901-b721-4b89-ba78-21c6342674c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:12:30,225::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:12:30,226::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:12:30,226::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9d9c2901-b721-4b89-ba78-21c6342674c4`::Granted request >Thread-34::DEBUG::2014-08-11 14:12:30,226::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a4266372-8fc4-4b3d-b765-c37ff5efed25`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:12:30,227::task::993::Storage.TaskManager.Task::(_decref) Task=`a4266372-8fc4-4b3d-b765-c37ff5efed25`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:12:30,247::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:30,247::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4266372-8fc4-4b3d-b765-c37ff5efed25`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:30,247::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4266372-8fc4-4b3d-b765-c37ff5efed25`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:12:30,248::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:12:30,248::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:12:30,248::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:12:30,249::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:12:30,249::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:12:30,249::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:12:30,249::task::993::Storage.TaskManager.Task::(_decref) Task=`a4266372-8fc4-4b3d-b765-c37ff5efed25`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:12:30,417::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:30,417::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:30,417::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:30,418::__init__::232::IOProcess::(_processLogs) (4571) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:30,418::__init__::232::IOProcess::(_processLogs) DEBUG|(4571) Queuing response >Thread-16::DEBUG::2014-08-11 14:12:30,420::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:12:30,437::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041621 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:12:30,438::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:30,439::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:30,439::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:30,439::__init__::232::IOProcess::(_processLogs) (4572) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:30,440::__init__::232::IOProcess::(_processLogs) DEBUG|(4572) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:30,441::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:30,442::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:30,443::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:30,443::__init__::232::IOProcess::(_processLogs) (4573) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:30,443::__init__::232::IOProcess::(_processLogs) (4573) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:30,444::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:30,445::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:30,446::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:30,448::__init__::232::IOProcess::(_processLogs) (4574) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:30,449::__init__::232::IOProcess::(_processLogs) (4574) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:30,451::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:30,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:30,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:30,455::__init__::232::IOProcess::(_processLogs) (4575) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:30,456::__init__::232::IOProcess::(_processLogs) DEBUG|(4575) Queuing response >Dummy-33::DEBUG::2014-08-11 14:12:31,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:31,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132929 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:33,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:33,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131099 s, 78.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:35,219::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:35,220::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:35,297::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:35,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:35,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132417 s, 77.3 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:12:36,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4ef4a3c-c7f9-4133-a093-fae877a255c7`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:36,302::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:12:36,303::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041621', 'lastCheck': '5.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:36,303::task::1191::Storage.TaskManager.Task::(prepare) Task=`b4ef4a3c-c7f9-4133-a093-fae877a255c7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041621', 'lastCheck': '5.8', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:36,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4ef4a3c-c7f9-4133-a093-fae877a255c7`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:36,304::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:36,304::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:36,304::task::993::Storage.TaskManager.Task::(_decref) Task=`b4ef4a3c-c7f9-4133-a093-fae877a255c7`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:12:36,323::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:37,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:37,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134254 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:39,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:39,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013292 s, 77.0 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:12:40,264::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:12:40,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9d090ac-b417-4d84-afb6-f784dbbac72c`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:40,296::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:40,305::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:40,307::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:40,310::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:12:40,310::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:40,312::task::1191::Storage.TaskManager.Task::(prepare) Task=`e9d090ac-b417-4d84-afb6-f784dbbac72c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:40,313::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9d090ac-b417-4d84-afb6-f784dbbac72c`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:40,313::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:40,314::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:40,314::task::993::Storage.TaskManager.Task::(_decref) Task=`e9d090ac-b417-4d84-afb6-f784dbbac72c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:12:40,324::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:12:40,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`30514ae2-5304-4231-a05e-2716c94d31ad`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:12:40,325::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:12:40,326::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0c99fac9-3d3c-45d5-8013-22046ef4b46f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:12:40,326::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:12:40,327::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:12:40,327::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0c99fac9-3d3c-45d5-8013-22046ef4b46f`::Granted request >Thread-34::DEBUG::2014-08-11 14:12:40,327::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`30514ae2-5304-4231-a05e-2716c94d31ad`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:12:40,328::task::993::Storage.TaskManager.Task::(_decref) Task=`30514ae2-5304-4231-a05e-2716c94d31ad`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:12:40,348::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:40,349::task::1191::Storage.TaskManager.Task::(prepare) Task=`30514ae2-5304-4231-a05e-2716c94d31ad`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:40,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`30514ae2-5304-4231-a05e-2716c94d31ad`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:12:40,349::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:12:40,350::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:12:40,350::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:12:40,350::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:12:40,350::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:12:40,351::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:12:40,351::task::993::Storage.TaskManager.Task::(_decref) Task=`30514ae2-5304-4231-a05e-2716c94d31ad`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:12:40,458::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:40,459::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:40,459::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:40,459::__init__::232::IOProcess::(_processLogs) (4576) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:40,460::__init__::232::IOProcess::(_processLogs) (4576) Queuing response >Thread-16::DEBUG::2014-08-11 14:12:40,460::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:12:40,478::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000522826 s, 870 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:12:40,479::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:40,480::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:40,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:40,480::__init__::232::IOProcess::(_processLogs) (4577) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:40,481::__init__::232::IOProcess::(_processLogs) DEBUG|(4577) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:40,482::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:40,483::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:40,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:40,484::__init__::232::IOProcess::(_processLogs) (4578) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:40,484::__init__::232::IOProcess::(_processLogs) (4578) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:40,484::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:40,485::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:40,486::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:40,486::__init__::232::IOProcess::(_processLogs) (4579) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:40,486::__init__::232::IOProcess::(_processLogs) (4579) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:40,486::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:40,487::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:40,488::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:40,488::__init__::232::IOProcess::(_processLogs) (4580) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:40,489::__init__::232::IOProcess::(_processLogs) (4580) Queuing response >Dummy-33::DEBUG::2014-08-11 14:12:41,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:42,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132741 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:44,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:44,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132126 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:45,319::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:45,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:45,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:46,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:46,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131924 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:48,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:48,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131756 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:50,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:50,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136243 s, 75.2 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:12:50,364::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-81::DEBUG::2014-08-11 14:12:50,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0a83ed3-1504-4b61-a393-cc1ac235e47e`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:50,396::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:50,406::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:50,408::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:50,409::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-81::INFO::2014-08-11 14:12:50,409::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:50,411::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0a83ed3-1504-4b61-a393-cc1ac235e47e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:12:50,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0a83ed3-1504-4b61-a393-cc1ac235e47e`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:50,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:50,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:50,413::task::993::Storage.TaskManager.Task::(_decref) Task=`d0a83ed3-1504-4b61-a393-cc1ac235e47e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:12:50,424::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:12:50,425::task::595::Storage.TaskManager.Task::(_updateState) Task=`47ea60a4-68d6-4cc1-b0db-e86fb0f5027b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:12:50,425::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:12:50,426::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5064f7f6-9e8b-43dc-acc2-db56477b0ee9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:12:50,426::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:12:50,426::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:12:50,427::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5064f7f6-9e8b-43dc-acc2-db56477b0ee9`::Granted request >Thread-34::DEBUG::2014-08-11 14:12:50,427::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`47ea60a4-68d6-4cc1-b0db-e86fb0f5027b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:12:50,427::task::993::Storage.TaskManager.Task::(_decref) Task=`47ea60a4-68d6-4cc1-b0db-e86fb0f5027b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:12:50,447::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:50,448::task::1191::Storage.TaskManager.Task::(prepare) Task=`47ea60a4-68d6-4cc1-b0db-e86fb0f5027b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:12:50,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`47ea60a4-68d6-4cc1-b0db-e86fb0f5027b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:12:50,448::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:12:50,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:12:50,449::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:12:50,449::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:12:50,449::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:12:50,449::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:12:50,450::task::993::Storage.TaskManager.Task::(_decref) Task=`47ea60a4-68d6-4cc1-b0db-e86fb0f5027b`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:12:50,489::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:50,489::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:50,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:50,490::__init__::232::IOProcess::(_processLogs) (4581) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:50,490::__init__::232::IOProcess::(_processLogs) (4581) Queuing response >Thread-16::DEBUG::2014-08-11 14:12:50,491::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:12:50,509::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000410323 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:12:50,510::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:50,510::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:50,511::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:50,511::__init__::232::IOProcess::(_processLogs) (4582) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:12:50,511::__init__::232::IOProcess::(_processLogs) (4582) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:50,513::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:50,514::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:50,514::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:50,514::__init__::232::IOProcess::(_processLogs) (4583) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:50,515::__init__::232::IOProcess::(_processLogs) (4583) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:50,515::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:50,516::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:50,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:50,517::__init__::232::IOProcess::(_processLogs) (4584) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:50,517::__init__::232::IOProcess::(_processLogs) (4584) Queuing response >Thread-14::DEBUG::2014-08-11 14:12:50,517::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:12:50,518::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:12:50,519::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:12:50,520::__init__::232::IOProcess::(_processLogs) (4585) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:12:50,520::__init__::232::IOProcess::(_processLogs) (4585) Queuing response >Thread-81::DEBUG::2014-08-11 14:12:51,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4a331a5-0e0c-434c-af04-d4a59c3af319`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:12:51,441::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-81::INFO::2014-08-11 14:12:51,442::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410323', 'lastCheck': '0.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:51,442::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4a331a5-0e0c-434c-af04-d4a59c3af319`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410323', 'lastCheck': '0.9', 'valid': True}} >Thread-81::DEBUG::2014-08-11 14:12:51,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4a331a5-0e0c-434c-af04-d4a59c3af319`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:12:51,443::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-08-11 14:12:51,443::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-08-11 14:12:51,443::task::993::Storage.TaskManager.Task::(_decref) Task=`d4a331a5-0e0c-434c-af04-d4a59c3af319`::ref 0 aborting False >Thread-81::DEBUG::2014-08-11 14:12:51,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:52,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:52,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131895 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:54,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:54,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132291 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:55,419::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:55,420::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:12:55,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:12:56,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:56,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132431 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:12:58,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:12:58,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131828 s, 77.7 MB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:13:00,462::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Dummy-33::DEBUG::2014-08-11 14:13:00,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-81::DEBUG::2014-08-11 14:13:00,464::task::595::Storage.TaskManager.Task::(_updateState) Task=`99cb6863-11af-4dcb-b8bf-8ef417a4188e`::moving from state init -> state preparing >Thread-81::INFO::2014-08-11 14:13:00,514::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:00,517::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:00,521::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:00,522::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:00,523::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:00,524::__init__::232::IOProcess::(_processLogs) (4586) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:00,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:00,527::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|(4586) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:00,528::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:13:00,531::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-81::INFO::2014-08-11 14:13:00,553::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-81::DEBUG::2014-08-11 14:13:00,558::task::1191::Storage.TaskManager.Task::(prepare) Task=`99cb6863-11af-4dcb-b8bf-8ef417a4188e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-16::DEBUG::2014-08-11 14:13:00,560::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00229965 s, 198 kB/s\n'; <rc> = 0 >Thread-81::DEBUG::2014-08-11 14:13:00,561::task::595::Storage.TaskManager.Task::(_updateState) Task=`99cb6863-11af-4dcb-b8bf-8ef417a4188e`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-08-11 14:13:00,562::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:13:00,565::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-81::DEBUG::2014-08-11 14:13:00,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:13:00,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 14:13:00,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0523767 s, 19.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:13:00,571::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-81::DEBUG::2014-08-11 14:13:00,572::task::993::Storage.TaskManager.Task::(_decref) Task=`99cb6863-11af-4dcb-b8bf-8ef417a4188e`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:13:00,575::__init__::232::IOProcess::(_processLogs) (4587) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:13:00,578::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|(4587) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:00,582::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:00,582::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:00,584::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:00,585::__init__::232::IOProcess::(_processLogs) (4588) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:00,587::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|(4588) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:00,589::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:00,591::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::DEBUG::2014-08-11 14:13:00,593::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:13:00,593::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:00,595::__init__::232::IOProcess::(_processLogs) (4589) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:13:00,596::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e41c568-081b-47fd-840b-2fea539a9464`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:13:00,598::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:13:00,598::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|(4589) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:00,601::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:00,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::DEBUG::2014-08-11 14:13:00,604::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`19bcfe46-79d2-4975-9dc7-73208cf486eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:13:00,606::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-14::DEBUG::2014-08-11 14:13:00,606::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:00,608::__init__::232::IOProcess::(_processLogs) (4590) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:13:00,608::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:13:00,610::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`19bcfe46-79d2-4975-9dc7-73208cf486eb`::Granted request >Thread-14::DEBUG::2014-08-11 14:13:00,610::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|(4590) Queuing response >Thread-34::DEBUG::2014-08-11 14:13:00,614::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5e41c568-081b-47fd-840b-2fea539a9464`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:13:00,616::task::993::Storage.TaskManager.Task::(_decref) Task=`5e41c568-081b-47fd-840b-2fea539a9464`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:13:00,662::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:00,663::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e41c568-081b-47fd-840b-2fea539a9464`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086243840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:00,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e41c568-081b-47fd-840b-2fea539a9464`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:13:00,665::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:13:00,666::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:13:00,668::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:13:00,670::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:13:00,671::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:13:00,672::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:13:00,674::task::993::Storage.TaskManager.Task::(_decref) Task=`5e41c568-081b-47fd-840b-2fea539a9464`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 14:13:00,798::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:48248 >Detector thread::DEBUG::2014-08-11 14:13:00,807::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:48248 >Detector thread::DEBUG::2014-08-11 14:13:00,807::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:48248 >Detector thread::DEBUG::2014-08-11 14:13:00,807::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 48248) >Dummy-33::DEBUG::2014-08-11 14:13:02,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:02,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0281748 s, 36.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:04,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:04,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131812 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:05,565::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:05,566::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:05,597::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-85::DEBUG::2014-08-11 14:13:06,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7ca77d2-8dad-4cd4-aed8-4eb79666cdc1`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:13:06,841::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-85::INFO::2014-08-11 14:13:06,841::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00229965', 'lastCheck': '6.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:06,842::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7ca77d2-8dad-4cd4-aed8-4eb79666cdc1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00229965', 'lastCheck': '6.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:06,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7ca77d2-8dad-4cd4-aed8-4eb79666cdc1`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:13:06,842::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:13:06,842::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:13:06,843::task::993::Storage.TaskManager.Task::(_decref) Task=`c7ca77d2-8dad-4cd4-aed8-4eb79666cdc1`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:13:06,861::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:06,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:07,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133495 s, 76.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:13:08,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`cafbe6bb-0619-4877-a410-dd3814ae437f`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:13:08,357::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:13:08,358::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:08,358::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:08,359::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:08,359::__init__::232::IOProcess::(_processLogs) (4591) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:13:08,360::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|(4591) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:08,360::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-55::INFO::2014-08-11 14:13:08,361::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:13:08,361::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-55::DEBUG::2014-08-11 14:13:08,362::task::1191::Storage.TaskManager.Task::(prepare) Task=`cafbe6bb-0619-4877-a410-dd3814ae437f`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:13:08,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`cafbe6bb-0619-4877-a410-dd3814ae437f`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:13:08,363::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:13:08,364::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:13:08,365::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:13:08,365::__init__::232::IOProcess::(_processLogs) (4592) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:13:08,366::task::993::Storage.TaskManager.Task::(_decref) Task=`cafbe6bb-0619-4877-a410-dd3814ae437f`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:13:08,367::__init__::232::IOProcess::(_processLogs) (4592) Queuing response >Dummy-33::DEBUG::2014-08-11 14:13:09,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:09,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132038 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:10,604::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:10,605::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:10,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:10,613::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:10,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:10,615::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:13:10,616::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:13:10,616::__init__::232::IOProcess::(_processLogs) (4593) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:13:10,633::__init__::232::IOProcess::(_processLogs) (4593) Queuing response >Thread-16::DEBUG::2014-08-11 14:13:10,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000457287 s, 995 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:13:10,636::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:10,636::__init__::232::IOProcess::(_processLogs) (4582) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:10,636::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:10,637::__init__::232::IOProcess::(_processLogs) (4594) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:13:10,637::__init__::232::IOProcess::(_processLogs) DEBUG|(4594) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:10,639::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:10,639::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:10,640::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:10,640::__init__::232::IOProcess::(_processLogs) (4595) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:10,640::__init__::232::IOProcess::(_processLogs) (4595) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:10,641::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:10,641::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:10,641::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:10,641::__init__::232::IOProcess::(_processLogs) (4596) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:10,642::__init__::232::IOProcess::(_processLogs) DEBUG|(4596) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:10,643::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:10,643::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:10,644::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:10,645::__init__::232::IOProcess::(_processLogs) (4597) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:10,645::__init__::232::IOProcess::(_processLogs) (4597) Queuing response >Thread-85::DEBUG::2014-08-11 14:13:10,691::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-85::DEBUG::2014-08-11 14:13:10,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`2daaa095-4de4-42de-8b69-3edfd1041a2e`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:13:10,692::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-85::INFO::2014-08-11 14:13:10,705::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:10,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`2daaa095-4de4-42de-8b69-3edfd1041a2e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:10,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`2daaa095-4de4-42de-8b69-3edfd1041a2e`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:13:10,705::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:13:10,706::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:13:10,706::task::993::Storage.TaskManager.Task::(_decref) Task=`2daaa095-4de4-42de-8b69-3edfd1041a2e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:13:10,717::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:13:10,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb5a3315-ecc7-4ae1-9f53-ddbaf183b52c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:13:10,718::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:13:10,718::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c202cd8f-dd98-4a7f-8e1f-24d741e46b2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:13:10,719::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:13:10,719::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:13:10,719::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c202cd8f-dd98-4a7f-8e1f-24d741e46b2e`::Granted request >Thread-34::DEBUG::2014-08-11 14:13:10,722::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`eb5a3315-ecc7-4ae1-9f53-ddbaf183b52c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:13:10,722::task::993::Storage.TaskManager.Task::(_decref) Task=`eb5a3315-ecc7-4ae1-9f53-ddbaf183b52c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:13:10,742::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086178304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:10,742::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb5a3315-ecc7-4ae1-9f53-ddbaf183b52c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086178304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:10,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb5a3315-ecc7-4ae1-9f53-ddbaf183b52c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:13:10,743::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:13:10,743::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:13:10,743::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:13:10,744::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:13:10,744::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:13:10,744::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:13:10,745::task::993::Storage.TaskManager.Task::(_decref) Task=`eb5a3315-ecc7-4ae1-9f53-ddbaf183b52c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:13:11,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:11,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131753 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:13,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:13,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013229 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:15,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:15,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131987 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:15,613::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:15,614::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:15,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:17,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:17,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132436 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:19,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:19,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250333 s, 40.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:13:20,646::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:20,646::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:20,698::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:13:20,699::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:13:20,699::__init__::232::IOProcess::(_processLogs) (4598) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:13:20,718::__init__::232::IOProcess::(_processLogs) (4598) Queuing response >Thread-16::DEBUG::2014-08-11 14:13:20,721::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000553327 s, 822 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:20,721::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:20,723::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:20,724::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:20,727::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:20,728::__init__::232::IOProcess::(_processLogs) (4599) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:20,729::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:20,730::__init__::232::IOProcess::(_processLogs) (4599) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:20,732::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:20,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:20,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:20,735::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:20,737::__init__::232::IOProcess::(_processLogs) (4600) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:20,739::__init__::232::IOProcess::(_processLogs) DEBUG|(4600) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:20,739::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:20,740::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:20,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:20,741::__init__::232::IOProcess::(_processLogs) (4601) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:20,741::__init__::232::IOProcess::(_processLogs) (4601) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:20,742::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:20,742::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:20,743::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:20,744::__init__::232::IOProcess::(_processLogs) (4602) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:20,744::__init__::232::IOProcess::(_processLogs) (4602) Queuing response >Thread-85::DEBUG::2014-08-11 14:13:20,757::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-85::DEBUG::2014-08-11 14:13:20,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`53134d67-e197-4eb2-9315-3918735189bf`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:13:20,758::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-85::INFO::2014-08-11 14:13:20,770::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:20,771::task::1191::Storage.TaskManager.Task::(prepare) Task=`53134d67-e197-4eb2-9315-3918735189bf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:20,771::task::595::Storage.TaskManager.Task::(_updateState) Task=`53134d67-e197-4eb2-9315-3918735189bf`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:13:20,771::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:13:20,772::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:13:20,772::task::993::Storage.TaskManager.Task::(_decref) Task=`53134d67-e197-4eb2-9315-3918735189bf`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:13:20,782::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:13:20,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6038ac7-56c8-435b-bf13-66a17b59721e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:13:20,783::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:13:20,784::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a7a7c22a-1647-4714-a9b8-9dc229e1a9af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:13:20,784::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:13:20,784::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:13:20,784::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a7a7c22a-1647-4714-a9b8-9dc229e1a9af`::Granted request >Thread-34::DEBUG::2014-08-11 14:13:20,785::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a6038ac7-56c8-435b-bf13-66a17b59721e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:13:20,785::task::993::Storage.TaskManager.Task::(_decref) Task=`a6038ac7-56c8-435b-bf13-66a17b59721e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:13:20,805::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086178304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:20,805::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6038ac7-56c8-435b-bf13-66a17b59721e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086178304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:20,806::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6038ac7-56c8-435b-bf13-66a17b59721e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:13:20,806::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:13:20,806::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:13:20,806::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:13:20,807::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:13:20,807::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:13:20,807::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:13:20,807::task::993::Storage.TaskManager.Task::(_decref) Task=`a6038ac7-56c8-435b-bf13-66a17b59721e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:13:21,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:21,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133241 s, 76.9 MB/s\n'; <rc> = 0 >Thread-85::DEBUG::2014-08-11 14:13:21,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb261b42-c541-43e6-b65b-8dff53c54a5b`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:13:21,947::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-85::INFO::2014-08-11 14:13:21,947::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000553327', 'lastCheck': '1.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:21,947::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb261b42-c541-43e6-b65b-8dff53c54a5b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000553327', 'lastCheck': '1.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:21,948::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb261b42-c541-43e6-b65b-8dff53c54a5b`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:13:21,948::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:13:21,948::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:13:21,948::task::993::Storage.TaskManager.Task::(_decref) Task=`fb261b42-c541-43e6-b65b-8dff53c54a5b`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:13:21,968::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:23,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:23,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132327 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:25,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:25,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131139 s, 78.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:25,741::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:25,742::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:25,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:27,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:27,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136948 s, 74.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:29,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:29,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131999 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:13:30,745::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:30,745::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:30,798::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:30,799::__init__::232::IOProcess::(_processLogs) (4603) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:13:30,799::__init__::232::IOProcess::(_processLogs) (4603) Queuing response >Thread-16::DEBUG::2014-08-11 14:13:30,800::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:30,818::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:13:30,819::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000517841 s, 879 kB/s\n'; <rc> = 0 >Thread-85::DEBUG::2014-08-11 14:13:30,822::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:13:30,824::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-85::DEBUG::2014-08-11 14:13:30,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a4673ec-94ee-4e16-b4de-863dea12b864`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:13:30,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-85::INFO::2014-08-11 14:13:30,828::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:13:30,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:30,830::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:30,831::__init__::232::IOProcess::(_processLogs) (4604) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:30,834::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:30,835::__init__::232::IOProcess::(_processLogs) DEBUG|(4604) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:30,839::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:30,840::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:30,840::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:30,841::__init__::232::IOProcess::(_processLogs) (4605) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:30,842::__init__::232::IOProcess::(_processLogs) (4605) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:30,843::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:30,843::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:30,844::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:30,845::__init__::232::IOProcess::(_processLogs) (4606) Got request for method 'access' >Thread-85::INFO::2014-08-11 14:13:30,845::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:13:30,846::__init__::232::IOProcess::(_processLogs) (4606) Queuing response >Thread-85::DEBUG::2014-08-11 14:13:30,847::task::1191::Storage.TaskManager.Task::(prepare) Task=`6a4673ec-94ee-4e16-b4de-863dea12b864`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:13:30,848::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-85::DEBUG::2014-08-11 14:13:30,848::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a4673ec-94ee-4e16-b4de-863dea12b864`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:13:30,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-85::DEBUG::2014-08-11 14:13:30,849::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:13:30,850::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-85::DEBUG::2014-08-11 14:13:30,851::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:13:30,851::__init__::232::IOProcess::(_processLogs) (4607) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:30,853::__init__::232::IOProcess::(_processLogs) DEBUG|(4607) Queuing response >Thread-85::DEBUG::2014-08-11 14:13:30,853::task::993::Storage.TaskManager.Task::(_decref) Task=`6a4673ec-94ee-4e16-b4de-863dea12b864`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:13:30,864::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:13:30,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`93e01955-3fba-41b6-a478-7646a5d0ed66`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:13:30,865::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:13:30,866::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a6e29cf0-daec-4c27-80ad-c090c7d0f1f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:13:30,866::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:13:30,867::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:13:30,867::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a6e29cf0-daec-4c27-80ad-c090c7d0f1f9`::Granted request >Thread-34::DEBUG::2014-08-11 14:13:30,868::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`93e01955-3fba-41b6-a478-7646a5d0ed66`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:13:30,868::task::993::Storage.TaskManager.Task::(_decref) Task=`93e01955-3fba-41b6-a478-7646a5d0ed66`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:13:30,888::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086178304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:30,888::task::1191::Storage.TaskManager.Task::(prepare) Task=`93e01955-3fba-41b6-a478-7646a5d0ed66`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086178304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:30,888::task::595::Storage.TaskManager.Task::(_updateState) Task=`93e01955-3fba-41b6-a478-7646a5d0ed66`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:13:30,889::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:13:30,889::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:13:30,889::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:13:30,890::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:13:30,890::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:13:30,890::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:13:30,890::task::993::Storage.TaskManager.Task::(_decref) Task=`93e01955-3fba-41b6-a478-7646a5d0ed66`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:13:31,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:31,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133732 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:33,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:33,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132374 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:35,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:35,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132193 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:35,845::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:35,846::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:35,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-85::DEBUG::2014-08-11 14:13:37,059::task::595::Storage.TaskManager.Task::(_updateState) Task=`93ec09f6-af11-4921-8c9c-12a19c945c83`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:13:37,060::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-85::INFO::2014-08-11 14:13:37,061::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000517841', 'lastCheck': '6.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:37,061::task::1191::Storage.TaskManager.Task::(prepare) Task=`93ec09f6-af11-4921-8c9c-12a19c945c83`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000517841', 'lastCheck': '6.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:37,062::task::595::Storage.TaskManager.Task::(_updateState) Task=`93ec09f6-af11-4921-8c9c-12a19c945c83`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:13:37,062::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:13:37,062::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:13:37,062::task::993::Storage.TaskManager.Task::(_decref) Task=`93ec09f6-af11-4921-8c9c-12a19c945c83`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:13:37,083::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:37,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:37,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134535 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:39,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:39,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131915 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:40,853::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:40,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:40,855::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:40,900::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:40,902::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:13:40,902::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:13:40,905::__init__::232::IOProcess::(_processLogs) (4608) Got request for method 'statvfs' >Thread-85::DEBUG::2014-08-11 14:13:40,923::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:40,925::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:40,925::__init__::232::IOProcess::(_processLogs) (4608) Queuing response >Thread-16::DEBUG::2014-08-11 14:13:40,927::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000655713 s, 694 kB/s\n'; <rc> = 0 >Thread-85::DEBUG::2014-08-11 14:13:40,931::task::595::Storage.TaskManager.Task::(_updateState) Task=`84b21ef3-853e-40fc-9af5-a2521ce79294`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:13:40,932::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-85::INFO::2014-08-11 14:13:40,933::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:13:40,934::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:40,936::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:40,937::__init__::232::IOProcess::(_processLogs) (4609) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:13:40,937::__init__::232::IOProcess::(_processLogs) (4609) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:40,937::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:13:40,938::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:40,938::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:40,939::__init__::232::IOProcess::(_processLogs) (4610) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:40,939::__init__::232::IOProcess::(_processLogs) (4610) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:40,941::__init__::232::IOProcess::(_processLogs) (4611) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:40,942::__init__::232::IOProcess::(_processLogs) (4612) Got request for method 'access' >Thread-85::INFO::2014-08-11 14:13:40,949::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:40,949::task::1191::Storage.TaskManager.Task::(prepare) Task=`84b21ef3-853e-40fc-9af5-a2521ce79294`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:40,950::task::595::Storage.TaskManager.Task::(_updateState) Task=`84b21ef3-853e-40fc-9af5-a2521ce79294`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:13:40,950::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:13:40,950::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:13:40,950::task::993::Storage.TaskManager.Task::(_decref) Task=`84b21ef3-853e-40fc-9af5-a2521ce79294`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:13:40,961::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:13:40,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b6b8e17-88e1-4fa6-b0b0-230e91afa97b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:13:40,962::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:13:40,963::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`42f38c67-52e7-4093-9ebb-57803f97e796`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:13:40,963::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:13:40,963::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:13:40,964::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`42f38c67-52e7-4093-9ebb-57803f97e796`::Granted request >Thread-34::DEBUG::2014-08-11 14:13:40,964::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2b6b8e17-88e1-4fa6-b0b0-230e91afa97b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:13:40,964::task::993::Storage.TaskManager.Task::(_decref) Task=`2b6b8e17-88e1-4fa6-b0b0-230e91afa97b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:13:40,984::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086112768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:40,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b6b8e17-88e1-4fa6-b0b0-230e91afa97b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086112768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:40,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b6b8e17-88e1-4fa6-b0b0-230e91afa97b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:13:40,985::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:13:40,985::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:13:40,986::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:13:40,986::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:13:40,986::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:13:40,986::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:13:40,987::task::993::Storage.TaskManager.Task::(_decref) Task=`2b6b8e17-88e1-4fa6-b0b0-230e91afa97b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:13:41,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:41,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132861 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:43,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:44,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132194 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:45,938::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:45,939::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:46,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:46,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:46,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132362 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:48,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:48,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132646 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:50,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:50,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132451 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:13:50,944::__init__::232::IOProcess::(_processLogs) (4613) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:13:50,945::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-85::DEBUG::2014-08-11 14:13:50,999::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-85::DEBUG::2014-08-11 14:13:51,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`000cce96-65f7-4334-847a-5daa174f6a5b`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 14:13:51,020::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000876145 s, 519 kB/s\n'; <rc> = 0 >Thread-85::INFO::2014-08-11 14:13:51,021::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:13:51,025::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:51,026::__init__::232::IOProcess::(_processLogs) (4614) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:51,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:13:51,027::__init__::232::IOProcess::(_processLogs) (4614) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:51,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:51,030::__init__::232::IOProcess::(_processLogs) (4615) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:51,032::__init__::232::IOProcess::(_processLogs) (4615) Queuing response >Thread-14::DEBUG::2014-08-11 14:13:51,034::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:13:51,035::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:13:51,036::__init__::232::IOProcess::(_processLogs) (4616) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:13:51,036::__init__::232::IOProcess::(_processLogs) (4616) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:51,037::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-85::INFO::2014-08-11 14:13:51,038::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:51,040::task::1191::Storage.TaskManager.Task::(prepare) Task=`000cce96-65f7-4334-847a-5daa174f6a5b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:13:51,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`000cce96-65f7-4334-847a-5daa174f6a5b`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:13:51,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-85::DEBUG::2014-08-11 14:13:51,044::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:13:51,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-85::DEBUG::2014-08-11 14:13:51,045::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:13:51,047::__init__::232::IOProcess::(_processLogs) (4617) Got request for method 'access' >Thread-85::DEBUG::2014-08-11 14:13:51,047::task::993::Storage.TaskManager.Task::(_decref) Task=`000cce96-65f7-4334-847a-5daa174f6a5b`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:13:51,048::__init__::232::IOProcess::(_processLogs) (4617) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:51,047::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:13:51,059::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:13:51,060::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ed5381b-040f-4649-978f-b2449a788289`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:13:51,060::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:13:51,061::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`61c190b7-2421-444a-b94d-39eb77860985`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:13:51,061::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:13:51,062::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:13:51,062::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`61c190b7-2421-444a-b94d-39eb77860985`::Granted request >Thread-34::DEBUG::2014-08-11 14:13:51,062::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2ed5381b-040f-4649-978f-b2449a788289`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:13:51,063::task::993::Storage.TaskManager.Task::(_decref) Task=`2ed5381b-040f-4649-978f-b2449a788289`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:13:51,082::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:51,083::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ed5381b-040f-4649-978f-b2449a788289`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:13:51,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ed5381b-040f-4649-978f-b2449a788289`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:13:51,083::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:13:51,083::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:13:51,084::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:13:51,084::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:13:51,084::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:13:51,084::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:13:51,085::task::993::Storage.TaskManager.Task::(_decref) Task=`2ed5381b-040f-4649-978f-b2449a788289`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:13:52,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`fcabe445-3ee0-48f6-a965-b25bdb4ecea4`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:13:52,181::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-85::INFO::2014-08-11 14:13:52,181::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000876145', 'lastCheck': '1.1', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:52,181::task::1191::Storage.TaskManager.Task::(prepare) Task=`fcabe445-3ee0-48f6-a965-b25bdb4ecea4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000876145', 'lastCheck': '1.1', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:13:52,182::task::595::Storage.TaskManager.Task::(_updateState) Task=`fcabe445-3ee0-48f6-a965-b25bdb4ecea4`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:13:52,182::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:13:52,182::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:13:52,182::task::993::Storage.TaskManager.Task::(_decref) Task=`fcabe445-3ee0-48f6-a965-b25bdb4ecea4`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:13:52,200::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:52,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:52,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147577 s, 69.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:54,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:54,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132266 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:56,057::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:56,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:13:56,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:13:56,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:56,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132128 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:13:58,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:13:58,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132579 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:00,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:00,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132597 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:14:01,044::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:01,045::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:01,045::__init__::232::IOProcess::(_processLogs) (4618) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:01,045::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4618) Queuing response >Thread-16::DEBUG::2014-08-11 14:14:01,046::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:14:01,064::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418749 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:14:01,066::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-85::DEBUG::2014-08-11 14:14:01,096::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:14:01,101::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-85::DEBUG::2014-08-11 14:14:01,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f5dbc63-12e6-492d-83b0-48412298eded`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:14:01,105::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:14:01,105::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:01,107::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:14:01,108::__init__::232::IOProcess::(_processLogs) (4619) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:01,109::__init__::232::IOProcess::(_processLogs) (4619) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:01,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:01,112::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:01,113::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:01,115::__init__::232::IOProcess::(_processLogs) (4620) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:01,116::__init__::232::IOProcess::(_processLogs) (4620) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:01,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:01,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:01,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:01,120::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:14:01,122::__init__::232::IOProcess::(_processLogs) (4621) Got request for method 'access' >Thread-85::INFO::2014-08-11 14:14:01,123::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:14:01,124::__init__::232::IOProcess::(_processLogs) (4621) Queuing response >Thread-85::DEBUG::2014-08-11 14:14:01,125::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f5dbc63-12e6-492d-83b0-48412298eded`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:14:01,127::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f5dbc63-12e6-492d-83b0-48412298eded`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:01,127::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-85::DEBUG::2014-08-11 14:14:01,128::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:14:01,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-85::DEBUG::2014-08-11 14:14:01,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:14:01,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-85::DEBUG::2014-08-11 14:14:01,133::task::993::Storage.TaskManager.Task::(_decref) Task=`1f5dbc63-12e6-492d-83b0-48412298eded`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:14:01,134::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:01,135::__init__::232::IOProcess::(_processLogs) (4622) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:01,136::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4622) Queuing response >Thread-34::DEBUG::2014-08-11 14:14:01,147::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:14:01,147::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b6bdfb6-c67f-4208-9336-89c2ed6f0e3a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:14:01,148::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:14:01,148::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3f2f041a-55be-41ed-9c62-1fc470e2f29a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:14:01,149::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:14:01,149::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:14:01,149::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3f2f041a-55be-41ed-9c62-1fc470e2f29a`::Granted request >Thread-34::DEBUG::2014-08-11 14:14:01,150::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0b6bdfb6-c67f-4208-9336-89c2ed6f0e3a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:14:01,150::task::993::Storage.TaskManager.Task::(_decref) Task=`0b6bdfb6-c67f-4208-9336-89c2ed6f0e3a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:14:01,170::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:01,171::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b6bdfb6-c67f-4208-9336-89c2ed6f0e3a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:01,171::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b6bdfb6-c67f-4208-9336-89c2ed6f0e3a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:14:01,171::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:14:01,171::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:14:01,172::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:14:01,172::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:14:01,172::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:14:01,172::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:14:01,173::task::993::Storage.TaskManager.Task::(_decref) Task=`0b6bdfb6-c67f-4208-9336-89c2ed6f0e3a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:14:02,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:02,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273498 s, 37.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:04,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:04,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210531 s, 48.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:06,139::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:06,140::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:06,200::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:06,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:06,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132225 s, 77.4 MB/s\n'; <rc> = 0 >Thread-85::DEBUG::2014-08-11 14:14:07,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`42b952d5-2f58-43b2-b517-0f53be39dfef`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:14:07,315::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-85::INFO::2014-08-11 14:14:07,316::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418749', 'lastCheck': '6.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:14:07,316::task::1191::Storage.TaskManager.Task::(prepare) Task=`42b952d5-2f58-43b2-b517-0f53be39dfef`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418749', 'lastCheck': '6.2', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:14:07,316::task::595::Storage.TaskManager.Task::(_updateState) Task=`42b952d5-2f58-43b2-b517-0f53be39dfef`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:14:07,316::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:14:07,317::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:14:07,317::task::993::Storage.TaskManager.Task::(_decref) Task=`42b952d5-2f58-43b2-b517-0f53be39dfef`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:14:07,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:14:08,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdeed90e-636b-4833-a2ee-36a5ac8394b3`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:14:08,480::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:14:08,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:08,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:08,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:08,482::__init__::232::IOProcess::(_processLogs) (4623) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:14:08,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4623) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:08,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:08,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:08,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:08,485::__init__::232::IOProcess::(_processLogs) (4624) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:14:08,485::__init__::232::IOProcess::(_processLogs) (4624) Queuing response >Thread-55::INFO::2014-08-11 14:14:08,485::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:14:08,486::task::1191::Storage.TaskManager.Task::(prepare) Task=`fdeed90e-636b-4833-a2ee-36a5ac8394b3`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:14:08,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdeed90e-636b-4833-a2ee-36a5ac8394b3`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:14:08,486::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:14:08,486::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:14:08,487::task::993::Storage.TaskManager.Task::(_decref) Task=`fdeed90e-636b-4833-a2ee-36a5ac8394b3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:14:08,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:08,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131708 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:10,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:10,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132063 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:14:11,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:11,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:11,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:11,139::__init__::232::IOProcess::(_processLogs) (4625) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:11,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4625) Queuing response >Thread-16::DEBUG::2014-08-11 14:14:11,140::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:14:11,158::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420849 s, 1.1 MB/s\n'; <rc> = 0 >Thread-85::DEBUG::2014-08-11 14:14:11,184::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:14:11,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-85::DEBUG::2014-08-11 14:14:11,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`407b05ac-969e-4ed2-a0c6-8462611b0187`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:14:11,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:11,204::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-85::INFO::2014-08-11 14:14:11,204::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:14:11,205::__init__::232::IOProcess::(_processLogs) (4626) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:11,209::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4626) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:11,210::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:11,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:11,215::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:14:11,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:11,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:11,218::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:11,219::__init__::232::IOProcess::(_processLogs) (4627) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:11,219::__init__::232::IOProcess::(_processLogs) (4627) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:11,221::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:11,221::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:11,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-85::INFO::2014-08-11 14:14:11,222::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:14:11,223::__init__::232::IOProcess::(_processLogs) (4628) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:11,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4628) Queuing response >Thread-85::DEBUG::2014-08-11 14:14:11,224::task::1191::Storage.TaskManager.Task::(prepare) Task=`407b05ac-969e-4ed2-a0c6-8462611b0187`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:14:11,225::task::595::Storage.TaskManager.Task::(_updateState) Task=`407b05ac-969e-4ed2-a0c6-8462611b0187`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:14:11,226::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:14:11,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:11,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-85::DEBUG::2014-08-11 14:14:11,227::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:14:11,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:11,229::__init__::232::IOProcess::(_processLogs) (4629) Got request for method 'access' >Thread-85::DEBUG::2014-08-11 14:14:11,229::task::993::Storage.TaskManager.Task::(_decref) Task=`407b05ac-969e-4ed2-a0c6-8462611b0187`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:14:11,230::__init__::232::IOProcess::(_processLogs) (4629) Queuing response >Thread-34::DEBUG::2014-08-11 14:14:11,243::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:14:11,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6eafa9d-b33b-4485-96d5-8ef9e18aa9fd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:14:11,244::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:14:11,245::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bad23c2e-f38d-4d85-b54d-599959691b5e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:14:11,245::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:14:11,246::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:14:11,246::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bad23c2e-f38d-4d85-b54d-599959691b5e`::Granted request >Thread-34::DEBUG::2014-08-11 14:14:11,247::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f6eafa9d-b33b-4485-96d5-8ef9e18aa9fd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:14:11,247::task::993::Storage.TaskManager.Task::(_decref) Task=`f6eafa9d-b33b-4485-96d5-8ef9e18aa9fd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:14:11,267::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:11,267::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6eafa9d-b33b-4485-96d5-8ef9e18aa9fd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:11,267::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6eafa9d-b33b-4485-96d5-8ef9e18aa9fd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:14:11,268::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:14:11,268::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:14:11,268::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:14:11,269::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:14:11,269::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:14:11,269::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:14:11,269::task::993::Storage.TaskManager.Task::(_decref) Task=`f6eafa9d-b33b-4485-96d5-8ef9e18aa9fd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:14:12,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:13,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131866 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:15,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:15,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134583 s, 76.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:16,219::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:16,220::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:16,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:17,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:17,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01319 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:19,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:19,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132536 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:21,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:14:21,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:21,236::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 14:14:21,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013306 s, 77.0 MB/s\n'; <rc> = 0 >Thread-85::DEBUG::2014-08-11 14:14:21,281::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:14:21,302::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Extracting request information... >Thread-85::DEBUG::2014-08-11 14:14:21,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`7dcd0306-ebd0-4cbf-8c2d-5e8b530ca0f2`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:14:21,305::__init__::232::IOProcess::(_processLogs) (4630) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:14:21,305::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-85::INFO::2014-08-11 14:14:21,307::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:14:21,328::__init__::232::IOProcess::(_processLogs) (4630) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:21,330::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:21,336::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:14:21,334::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000684032 s, 665 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:21,341::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-85::INFO::2014-08-11 14:14:21,345::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:14:21,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-85::DEBUG::2014-08-11 14:14:21,347::task::1191::Storage.TaskManager.Task::(prepare) Task=`7dcd0306-ebd0-4cbf-8c2d-5e8b530ca0f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:14:21,349::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-85::DEBUG::2014-08-11 14:14:21,350::task::595::Storage.TaskManager.Task::(_updateState) Task=`7dcd0306-ebd0-4cbf-8c2d-5e8b530ca0f2`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:14:21,352::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:21,353::__init__::232::IOProcess::(_processLogs) (4631) Got request for method 'statvfs' >Thread-85::DEBUG::2014-08-11 14:14:21,353::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:14:21,355::__init__::232::IOProcess::(_processLogs) (4631) Queuing response >Thread-85::DEBUG::2014-08-11 14:14:21,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:14:21,357::task::993::Storage.TaskManager.Task::(_decref) Task=`7dcd0306-ebd0-4cbf-8c2d-5e8b530ca0f2`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:14:21,357::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:21,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:21,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:21,369::__init__::232::IOProcess::(_processLogs) (4632) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:21,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(4632) Queuing response >Thread-34::DEBUG::2014-08-11 14:14:21,379::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:14:21,382::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-34::DEBUG::2014-08-11 14:14:21,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`bbe34177-ea59-42fe-8a84-3d1b24ee7cd4`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:14:21,385::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::INFO::2014-08-11 14:14:21,388::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:14:21,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:21,392::__init__::232::IOProcess::(_processLogs) (4633) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:14:21,397::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5132c809-3705-4cc5-a087-5e2f12d578a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-14::DEBUG::2014-08-11 14:14:21,398::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(4633) Queuing response >Thread-34::DEBUG::2014-08-11 14:14:21,399::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:14:21,403::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:14:21,407::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5132c809-3705-4cc5-a087-5e2f12d578a9`::Granted request >Thread-14::DEBUG::2014-08-11 14:14:21,407::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:21,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::DEBUG::2014-08-11 14:14:21,415::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bbe34177-ea59-42fe-8a84-3d1b24ee7cd4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-14::DEBUG::2014-08-11 14:14:21,416::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-34::DEBUG::2014-08-11 14:14:21,417::task::993::Storage.TaskManager.Task::(_decref) Task=`bbe34177-ea59-42fe-8a84-3d1b24ee7cd4`::ref 1 aborting False >Thread-14::DEBUG::2014-08-11 14:14:21,419::__init__::232::IOProcess::(_processLogs) (4634) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:21,422::__init__::232::IOProcess::(_processLogs) (4634) Queuing response >Thread-34::INFO::2014-08-11 14:14:21,506::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:21,508::task::1191::Storage.TaskManager.Task::(prepare) Task=`bbe34177-ea59-42fe-8a84-3d1b24ee7cd4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:21,509::task::595::Storage.TaskManager.Task::(_updateState) Task=`bbe34177-ea59-42fe-8a84-3d1b24ee7cd4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:14:21,511::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:14:21,512::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:14:21,514::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:14:21,517::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:14:21,518::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:14:21,519::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:14:21,521::task::993::Storage.TaskManager.Task::(_decref) Task=`bbe34177-ea59-42fe-8a84-3d1b24ee7cd4`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:14:22,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`522cab27-c345-4048-a05e-78a9a91d6fbe`::moving from state init -> state preparing >Thread-85::INFO::2014-08-11 14:14:22,441::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-85::INFO::2014-08-11 14:14:22,442::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684032', 'lastCheck': '1.0', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:14:22,442::task::1191::Storage.TaskManager.Task::(prepare) Task=`522cab27-c345-4048-a05e-78a9a91d6fbe`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000684032', 'lastCheck': '1.0', 'valid': True}} >Thread-85::DEBUG::2014-08-11 14:14:22,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`522cab27-c345-4048-a05e-78a9a91d6fbe`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:14:22,442::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:14:22,443::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:14:22,443::task::993::Storage.TaskManager.Task::(_decref) Task=`522cab27-c345-4048-a05e-78a9a91d6fbe`::ref 0 aborting False >Thread-85::DEBUG::2014-08-11 14:14:22,474::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:23,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:23,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133272 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:25,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:25,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132892 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:26,349::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:26,351::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:26,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:27,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:27,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013108 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:29,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:29,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134021 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:31,405::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:31,406::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:14:31,408::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:31,502::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:31,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:14:31,507::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:14:31,507::__init__::232::IOProcess::(_processLogs) (4635) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:31,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:14:31,527::__init__::232::IOProcess::(_processLogs) (4635) Queuing response >Thread-16::DEBUG::2014-08-11 14:14:31,530::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000689852 s, 660 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:14:31,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:31,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:31,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:31,532::__init__::232::IOProcess::(_processLogs) (4636) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:31,533::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(4636) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:31,534::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:31,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:31,536::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:31,536::__init__::232::IOProcess::(_processLogs) (4637) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:31,536::__init__::232::IOProcess::(_processLogs) (4637) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:31,536::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:31,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-85::DEBUG::2014-08-11 14:14:31,540::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:14:31,540::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:31,542::__init__::232::IOProcess::(_processLogs) (4638) Got request for method 'access' >Thread-85::DEBUG::2014-08-11 14:14:31,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`636851da-f2cf-4005-b08e-0a97d07a3c57`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:14:31,544::__init__::232::IOProcess::(_processLogs) (4638) Queuing response >Thread-85::INFO::2014-08-11 14:14:31,544::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:14:31,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:31,547::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:31,548::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:31,548::__init__::232::IOProcess::(_processLogs) (4639) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:31,549::__init__::232::IOProcess::(_processLogs) (4639) Queuing response >Thread-85::INFO::2014-08-11 14:14:31,559::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:14:31,559::task::1191::Storage.TaskManager.Task::(prepare) Task=`636851da-f2cf-4005-b08e-0a97d07a3c57`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-85::DEBUG::2014-08-11 14:14:31,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`636851da-f2cf-4005-b08e-0a97d07a3c57`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-08-11 14:14:31,560::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-08-11 14:14:31,561::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-08-11 14:14:31,561::task::993::Storage.TaskManager.Task::(_decref) Task=`636851da-f2cf-4005-b08e-0a97d07a3c57`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 14:14:31,562::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:36227 >Detector thread::DEBUG::2014-08-11 14:14:31,563::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:36227 >Detector thread::DEBUG::2014-08-11 14:14:31,563::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:36227 >Detector thread::DEBUG::2014-08-11 14:14:31,563::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 36227) >Thread-34::DEBUG::2014-08-11 14:14:31,583::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:14:31,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`9497e5f2-f780-455d-8f5d-f9bb6dce4852`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:14:31,584::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:14:31,584::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3ce693f9-f006-4bd6-932b-49048333641d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:14:31,585::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:14:31,585::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:14:31,585::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3ce693f9-f006-4bd6-932b-49048333641d`::Granted request >Thread-34::DEBUG::2014-08-11 14:14:31,586::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9497e5f2-f780-455d-8f5d-f9bb6dce4852`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:14:31,586::task::993::Storage.TaskManager.Task::(_decref) Task=`9497e5f2-f780-455d-8f5d-f9bb6dce4852`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:14:31,606::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:31,607::task::1191::Storage.TaskManager.Task::(prepare) Task=`9497e5f2-f780-455d-8f5d-f9bb6dce4852`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:31,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`9497e5f2-f780-455d-8f5d-f9bb6dce4852`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:14:31,607::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:14:31,607::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:14:31,608::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:14:31,608::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:14:31,608::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:14:31,608::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:14:31,609::task::993::Storage.TaskManager.Task::(_decref) Task=`9497e5f2-f780-455d-8f5d-f9bb6dce4852`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:14:31,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:31,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133445 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:33,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:33,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135557 s, 75.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:35,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:36,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132025 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:36,537::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:36,538::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:36,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:14:37,610::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bbda673-eb62-4a7f-acef-745895fdf289`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:14:37,611::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:14:37,611::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000689852', 'lastCheck': '6.1', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:14:37,611::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bbda673-eb62-4a7f-acef-745895fdf289`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000689852', 'lastCheck': '6.1', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:14:37,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bbda673-eb62-4a7f-acef-745895fdf289`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:14:37,612::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:14:37,612::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:14:37,612::task::993::Storage.TaskManager.Task::(_decref) Task=`8bbda673-eb62-4a7f-acef-745895fdf289`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:14:37,632::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:38,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:38,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131754 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:40,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:40,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131679 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:14:41,549::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:41,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:41,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:41,602::__init__::232::IOProcess::(_processLogs) (4640) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:41,602::__init__::232::IOProcess::(_processLogs) (4640) Queuing response >Thread-16::DEBUG::2014-08-11 14:14:41,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:41,621::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:14:41,623::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00047021 s, 968 kB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:14:41,626::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:14:41,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:41,629::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:14:41,631::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:41,634::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:14:41,633::task::595::Storage.TaskManager.Task::(_updateState) Task=`193fe5ae-52f1-4535-9b1a-47de0625c2f8`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:14:41,635::__init__::232::IOProcess::(_processLogs) (4641) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:41,637::__init__::232::IOProcess::(_processLogs) (4641) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:41,638::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:14:41,638::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:14:41,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:41,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:41,643::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:41,644::__init__::232::IOProcess::(_processLogs) (4642) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:41,644::__init__::232::IOProcess::(_processLogs) (4642) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:41,644::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:41,645::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:41,646::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:41,646::__init__::232::IOProcess::(_processLogs) (4643) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:41,646::__init__::232::IOProcess::(_processLogs) (4643) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:41,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:41,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:41,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:41,649::__init__::232::IOProcess::(_processLogs) (4644) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:41,650::__init__::232::IOProcess::(_processLogs) (4644) Queuing response >Thread-86::INFO::2014-08-11 14:14:41,656::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:14:41,656::task::1191::Storage.TaskManager.Task::(prepare) Task=`193fe5ae-52f1-4535-9b1a-47de0625c2f8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:14:41,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`193fe5ae-52f1-4535-9b1a-47de0625c2f8`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:14:41,657::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:14:41,657::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:14:41,657::task::993::Storage.TaskManager.Task::(_decref) Task=`193fe5ae-52f1-4535-9b1a-47de0625c2f8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:14:41,667::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:14:41,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd6f90fb-92d2-4e53-963c-547700bc02a9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:14:41,668::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:14:41,669::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3519ceaa-50a2-44ac-8006-7407ba364447`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:14:41,669::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:14:41,669::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:14:41,670::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3519ceaa-50a2-44ac-8006-7407ba364447`::Granted request >Thread-34::DEBUG::2014-08-11 14:14:41,670::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dd6f90fb-92d2-4e53-963c-547700bc02a9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:14:41,670::task::993::Storage.TaskManager.Task::(_decref) Task=`dd6f90fb-92d2-4e53-963c-547700bc02a9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:14:41,690::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:41,690::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd6f90fb-92d2-4e53-963c-547700bc02a9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:41,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd6f90fb-92d2-4e53-963c-547700bc02a9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:14:41,691::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:14:41,691::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:14:41,691::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:14:41,692::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:14:41,692::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:14:41,692::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:14:41,692::task::993::Storage.TaskManager.Task::(_decref) Task=`dd6f90fb-92d2-4e53-963c-547700bc02a9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:14:42,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:42,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131816 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:44,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:44,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131851 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:46,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:46,289::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013164 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:46,648::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:46,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:46,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:48,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:48,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131422 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:50,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:50,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133677 s, 76.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:14:51,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:51,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:51,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:51,702::__init__::232::IOProcess::(_processLogs) (4645) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:51,702::__init__::232::IOProcess::(_processLogs) (4645) Queuing response >Thread-16::DEBUG::2014-08-11 14:14:51,703::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:14:51,720::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:14:51,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`636ffc05-b6dd-496c-bf56-1e74f3dc7bb1`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:14:51,722::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:51,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:51,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:51,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:14:51,729::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423749 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:14:51,731::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:51,731::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:51,732::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:51,732::__init__::232::IOProcess::(_processLogs) (4646) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:14:51,735::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4646) Queuing response >Thread-86::INFO::2014-08-11 14:14:51,735::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:14:51,737::task::1191::Storage.TaskManager.Task::(prepare) Task=`636ffc05-b6dd-496c-bf56-1e74f3dc7bb1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:14:51,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:14:51,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`636ffc05-b6dd-496c-bf56-1e74f3dc7bb1`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:14:51,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:14:51,740::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:14:51,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:14:51,742::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:14:51,743::__init__::232::IOProcess::(_processLogs) (4647) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:14:51,743::task::993::Storage.TaskManager.Task::(_decref) Task=`636ffc05-b6dd-496c-bf56-1e74f3dc7bb1`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:14:51,744::__init__::232::IOProcess::(_processLogs) (4647) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:51,746::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:51,746::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:51,747::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:51,747::__init__::232::IOProcess::(_processLogs) (4648) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:51,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4648) Queuing response >Thread-14::DEBUG::2014-08-11 14:14:51,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:14:51,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:14:51,750::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:14:51,750::__init__::232::IOProcess::(_processLogs) (4649) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:14:51,751::__init__::232::IOProcess::(_processLogs) (4649) Queuing response >Thread-34::DEBUG::2014-08-11 14:14:51,756::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:14:51,757::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb59a798-303a-4861-ba03-2adfbc4c9bff`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:14:51,757::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:14:51,758::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f92922e2-e561-44e1-98ca-76c83345b494`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:14:51,758::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:14:51,759::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:14:51,759::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f92922e2-e561-44e1-98ca-76c83345b494`::Granted request >Thread-34::DEBUG::2014-08-11 14:14:51,759::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cb59a798-303a-4861-ba03-2adfbc4c9bff`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:14:51,760::task::993::Storage.TaskManager.Task::(_decref) Task=`cb59a798-303a-4861-ba03-2adfbc4c9bff`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:14:51,779::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:51,780::task::1191::Storage.TaskManager.Task::(prepare) Task=`cb59a798-303a-4861-ba03-2adfbc4c9bff`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:14:51,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb59a798-303a-4861-ba03-2adfbc4c9bff`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:14:51,780::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:14:51,780::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:14:51,781::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:14:51,781::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:14:51,781::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:14:51,781::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:14:51,782::task::993::Storage.TaskManager.Task::(_decref) Task=`cb59a798-303a-4861-ba03-2adfbc4c9bff`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:14:52,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:52,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132047 s, 77.5 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:14:52,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`dceab442-40c7-4d8c-acda-67e710001737`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:14:52,737::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:14:52,738::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423749', 'lastCheck': '1.0', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:14:52,738::task::1191::Storage.TaskManager.Task::(prepare) Task=`dceab442-40c7-4d8c-acda-67e710001737`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423749', 'lastCheck': '1.0', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:14:52,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`dceab442-40c7-4d8c-acda-67e710001737`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:14:52,738::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:14:52,739::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:14:52,739::task::993::Storage.TaskManager.Task::(_decref) Task=`dceab442-40c7-4d8c-acda-67e710001737`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:14:52,758::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:54,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:54,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132395 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:14:56,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:56,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132592 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:56,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:56,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:14:56,802::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:14:58,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:14:58,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132197 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:00,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:00,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132544 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:01,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:15:01,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:15:01,794::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:15:01,802::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:01,804::__init__::232::IOProcess::(_processLogs) (4650) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:15:01,804::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a4c38f1-509f-4a7a-b940-99e686e0e0db`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:01,806::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 14:15:01,807::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::INFO::2014-08-11 14:15:01,828::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:01,830::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:15:01,834::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000552462 s, 824 kB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:15:01,834::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a4c38f1-509f-4a7a-b940-99e686e0e0db`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:01,838::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a4c38f1-509f-4a7a-b940-99e686e0e0db`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:15:01,840::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:15:01,842::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:15:01,844::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:15:01,845::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:15:01,847::__init__::232::IOProcess::(_processLogs) (4651) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:15:01,849::task::993::Storage.TaskManager.Task::(_decref) Task=`5a4c38f1-509f-4a7a-b940-99e686e0e0db`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:01,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:15:01,857::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:01,861::__init__::232::IOProcess::(_processLogs) (4652) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:01,864::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:15:01,865::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:01,868::__init__::232::IOProcess::(_processLogs) (4653) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:01,873::__init__::232::IOProcess::(_processLogs) (4653) Queuing response >Thread-34::DEBUG::2014-08-11 14:15:01,873::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:15:01,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`046d6f16-818d-442d-ab09-cffe17c0409b`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:15:01,879::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::INFO::2014-08-11 14:15:01,880::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:15:01,881::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:01,882::__init__::232::IOProcess::(_processLogs) (4654) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:15:01,884::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`68b44186-e688-4505-befc-8620db30361a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-14::DEBUG::2014-08-11 14:15:01,884::__init__::232::IOProcess::(_processLogs) (4654) Queuing response >Thread-34::DEBUG::2014-08-11 14:15:01,884::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:15:01,886::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:15:01,889::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`68b44186-e688-4505-befc-8620db30361a`::Granted request >Thread-34::DEBUG::2014-08-11 14:15:01,890::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`046d6f16-818d-442d-ab09-cffe17c0409b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:15:01,890::task::993::Storage.TaskManager.Task::(_decref) Task=`046d6f16-818d-442d-ab09-cffe17c0409b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:15:01,910::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:01,911::task::1191::Storage.TaskManager.Task::(prepare) Task=`046d6f16-818d-442d-ab09-cffe17c0409b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863086047232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:01,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`046d6f16-818d-442d-ab09-cffe17c0409b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:15:01,911::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:15:01,912::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:15:01,912::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:15:01,912::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:15:01,913::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:15:01,913::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:15:01,913::task::993::Storage.TaskManager.Task::(_decref) Task=`046d6f16-818d-442d-ab09-cffe17c0409b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:15:02,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:02,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0285062 s, 35.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:04,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:04,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133239 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:06,878::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:06,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:06,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:07,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:07,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134076 s, 76.4 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:15:07,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`e970afad-710f-408b-81c4-353e72118143`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:07,875::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:15:07,876::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000552462', 'lastCheck': '6.0', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:07,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`e970afad-710f-408b-81c4-353e72118143`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000552462', 'lastCheck': '6.0', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:07,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`e970afad-710f-408b-81c4-353e72118143`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:07,877::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:07,877::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:07,877::task::993::Storage.TaskManager.Task::(_decref) Task=`e970afad-710f-408b-81c4-353e72118143`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:15:07,902::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:15:08,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`3319393a-7fcc-4438-afaa-fd89764d6d5b`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:15:08,637::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:15:08,638::__init__::232::IOProcess::(_processLogs) (4655) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:15:08,640::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:08,640::__init__::232::IOProcess::(_processLogs) (4656) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:15:08,641::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:15:08,641::task::1191::Storage.TaskManager.Task::(prepare) Task=`3319393a-7fcc-4438-afaa-fd89764d6d5b`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:15:08,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`3319393a-7fcc-4438-afaa-fd89764d6d5b`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:15:08,641::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:15:08,641::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:15:08,642::task::993::Storage.TaskManager.Task::(_decref) Task=`3319393a-7fcc-4438-afaa-fd89764d6d5b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:15:09,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:09,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132518 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:11,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:11,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132269 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:11,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:11,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:15:11,890::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:11,902::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:11,904::__init__::232::IOProcess::(_processLogs) (4657) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:11,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:15:11,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:15:11,931::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-16::DEBUG::2014-08-11 14:15:11,932::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000438812 s, 1.0 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:15:11,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`621d0220-900b-4de8-9751-d3508a35e110`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:15:11,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::INFO::2014-08-11 14:15:11,939::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:15:11,940::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:11,944::__init__::232::IOProcess::(_processLogs) (4658) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:15:11,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:11,948::__init__::232::IOProcess::(_processLogs) (4659) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:11,948::__init__::232::IOProcess::(_processLogs) (4659) Queuing response >Thread-14::DEBUG::2014-08-11 14:15:11,950::__init__::232::IOProcess::(_processLogs) (4660) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:15:11,956::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:15:11,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:15:11,958::task::1191::Storage.TaskManager.Task::(prepare) Task=`621d0220-900b-4de8-9751-d3508a35e110`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:15:11,960::__init__::232::IOProcess::(_processLogs) (4661) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:15:11,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`621d0220-900b-4de8-9751-d3508a35e110`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:15:11,961::__init__::232::IOProcess::(_processLogs) (4661) Queuing response >Thread-86::DEBUG::2014-08-11 14:15:11,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:11,963::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:11,963::task::993::Storage.TaskManager.Task::(_decref) Task=`621d0220-900b-4de8-9751-d3508a35e110`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:15:11,973::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:15:11,974::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ee6db9e-b791-48e5-af9e-c398815e455e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:15:11,974::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:15:11,975::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5f5737ac-e888-4dc0-9398-d44bfd605b99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:15:11,975::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:15:11,975::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:15:11,976::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5f5737ac-e888-4dc0-9398-d44bfd605b99`::Granted request >Thread-34::DEBUG::2014-08-11 14:15:11,976::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9ee6db9e-b791-48e5-af9e-c398815e455e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:15:11,976::task::993::Storage.TaskManager.Task::(_decref) Task=`9ee6db9e-b791-48e5-af9e-c398815e455e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:15:11,996::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085981696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:11,997::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ee6db9e-b791-48e5-af9e-c398815e455e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085981696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:11,997::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ee6db9e-b791-48e5-af9e-c398815e455e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:15:11,997::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:15:11,997::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:15:11,998::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:15:11,998::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:15:11,998::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:15:11,998::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:15:11,999::task::993::Storage.TaskManager.Task::(_decref) Task=`9ee6db9e-b791-48e5-af9e-c398815e455e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:15:13,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:13,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133032 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:15,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:15,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013283 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:16,914::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:16,915::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:17,002::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:17,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:17,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133903 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:19,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:19,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133196 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:21,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:21,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132782 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:21,961::__init__::232::IOProcess::(_processLogs) (4662) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:15:21,962::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:22,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:22,022::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:22,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:15:22,026::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-16::DEBUG::2014-08-11 14:15:22,027::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00546213 s, 83.3 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:22,030::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:15:22,029::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f91fb25-f492-486f-a9d9-f711cc0e496c`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:15:22,031::__init__::232::IOProcess::(_processLogs) (4663) Got request for method 'statvfs' >Thread-86::INFO::2014-08-11 14:15:22,032::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:15:22,034::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:22,035::__init__::232::IOProcess::(_processLogs) (4664) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:22,036::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:22,036::__init__::232::IOProcess::(_processLogs) (4665) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:22,036::__init__::232::IOProcess::(_processLogs) (4665) Queuing response >Thread-14::DEBUG::2014-08-11 14:15:22,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:22,038::__init__::232::IOProcess::(_processLogs) (4666) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:15:22,049::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:22,049::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f91fb25-f492-486f-a9d9-f711cc0e496c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:22,049::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f91fb25-f492-486f-a9d9-f711cc0e496c`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:22,050::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:22,050::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:22,050::task::993::Storage.TaskManager.Task::(_decref) Task=`1f91fb25-f492-486f-a9d9-f711cc0e496c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:15:22,060::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:15:22,061::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cff3128-a704-42c4-8afe-5e6974202142`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:15:22,061::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:15:22,062::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`021a1a52-c09b-4c5d-8b9a-45be4edf1acf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:15:22,062::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:15:22,062::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:15:22,063::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`021a1a52-c09b-4c5d-8b9a-45be4edf1acf`::Granted request >Thread-34::DEBUG::2014-08-11 14:15:22,063::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3cff3128-a704-42c4-8afe-5e6974202142`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:15:22,063::task::993::Storage.TaskManager.Task::(_decref) Task=`3cff3128-a704-42c4-8afe-5e6974202142`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:15:22,083::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085981696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:22,083::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cff3128-a704-42c4-8afe-5e6974202142`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085981696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:22,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cff3128-a704-42c4-8afe-5e6974202142`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:15:22,084::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:15:22,084::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:15:22,084::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:15:22,085::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:15:22,085::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:15:22,085::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:15:22,085::task::993::Storage.TaskManager.Task::(_decref) Task=`3cff3128-a704-42c4-8afe-5e6974202142`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:15:23,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cd52211-cf14-4c17-843f-32ecfebb6e81`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:23,003::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:15:23,003::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00546213', 'lastCheck': '1.0', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:23,003::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cd52211-cf14-4c17-843f-32ecfebb6e81`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00546213', 'lastCheck': '1.0', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:23,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cd52211-cf14-4c17-843f-32ecfebb6e81`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:23,004::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:23,004::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:23,004::task::993::Storage.TaskManager.Task::(_decref) Task=`3cd52211-cf14-4c17-843f-32ecfebb6e81`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:15:23,022::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:23,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:23,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132508 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:25,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:25,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132218 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:27,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:27,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:27,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:27,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:27,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132139 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:29,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:29,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132396 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:31,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:31,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013418 s, 76.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:32,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:32,045::__init__::232::IOProcess::(_processLogs) (4667) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:15:32,048::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:15:32,101::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:15:32,102::__init__::232::IOProcess::(_processLogs) (4667) Queuing response >Thread-86::DEBUG::2014-08-11 14:15:32,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2ca2a37-1664-47b0-9a97-8a58b5983e83`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:32,128::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:32,128::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:15:32,129::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000921115 s, 494 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:32,133::__init__::232::IOProcess::(_processLogs) (4668) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:15:32,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:32,143::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:15:32,145::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:32,145::__init__::232::IOProcess::(_processLogs) (4669) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:15:32,146::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:15:32,148::__init__::232::IOProcess::(_processLogs) (4669) Queuing response >Thread-86::DEBUG::2014-08-11 14:15:32,151::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2ca2a37-1664-47b0-9a97-8a58b5983e83`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:32,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2ca2a37-1664-47b0-9a97-8a58b5983e83`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:32,155::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:32,156::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:15:32,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:15:32,159::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:15:32,161::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:15:32,163::task::993::Storage.TaskManager.Task::(_decref) Task=`f2ca2a37-1664-47b0-9a97-8a58b5983e83`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:15:32,164::__init__::232::IOProcess::(_processLogs) (4670) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:32,168::__init__::232::IOProcess::(_processLogs) (4670) Queuing response >Thread-14::DEBUG::2014-08-11 14:15:32,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:32,171::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:32,172::__init__::232::IOProcess::(_processLogs) (4671) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:32,173::__init__::232::IOProcess::(_processLogs) (4671) Queuing response >Thread-34::DEBUG::2014-08-11 14:15:32,179::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:15:32,182::task::595::Storage.TaskManager.Task::(_updateState) Task=`abc1470b-c470-4f0b-8b60-554b8e4ab955`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:15:32,183::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:15:32,186::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6793fb5f-5ccc-4df7-aa48-ed6dac040fa7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:15:32,187::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:15:32,189::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:15:32,190::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6793fb5f-5ccc-4df7-aa48-ed6dac040fa7`::Granted request >Thread-34::DEBUG::2014-08-11 14:15:32,194::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`abc1470b-c470-4f0b-8b60-554b8e4ab955`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:15:32,195::task::993::Storage.TaskManager.Task::(_decref) Task=`abc1470b-c470-4f0b-8b60-554b8e4ab955`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:15:32,250::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085981696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:32,251::task::1191::Storage.TaskManager.Task::(prepare) Task=`abc1470b-c470-4f0b-8b60-554b8e4ab955`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085981696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:32,252::task::595::Storage.TaskManager.Task::(_updateState) Task=`abc1470b-c470-4f0b-8b60-554b8e4ab955`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:15:32,253::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:15:32,254::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:15:32,256::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:15:32,258::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:15:32,259::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:15:32,260::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:15:32,262::task::993::Storage.TaskManager.Task::(_decref) Task=`abc1470b-c470-4f0b-8b60-554b8e4ab955`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:15:33,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:33,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01333 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:35,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:35,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132026 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:37,172::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:37,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:37,203::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:37,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:37,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134245 s, 76.3 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:15:38,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`24842de8-f673-49ee-8df4-e036efa94e15`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:38,116::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:15:38,117::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000921115', 'lastCheck': '5.9', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:38,117::task::1191::Storage.TaskManager.Task::(prepare) Task=`24842de8-f673-49ee-8df4-e036efa94e15`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000921115', 'lastCheck': '5.9', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:38,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`24842de8-f673-49ee-8df4-e036efa94e15`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:38,117::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:38,118::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:38,118::task::993::Storage.TaskManager.Task::(_decref) Task=`24842de8-f673-49ee-8df4-e036efa94e15`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:15:38,138::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:39,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:39,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131771 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:41,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:41,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131376 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:42,177::__init__::232::IOProcess::(_processLogs) (4672) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:15:42,178::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:42,220::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:42,222::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:42,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:15:42,225::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416492 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:42,226::__init__::232::IOProcess::(_processLogs) (4673) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:15:42,228::__init__::232::IOProcess::(_processLogs) (4674) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:42,229::__init__::232::IOProcess::(_processLogs) (4675) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:42,230::__init__::232::IOProcess::(_processLogs) (4676) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:15:42,281::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:15:42,282::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7db64b1-1858-46d3-a7d5-9276876c2672`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:42,283::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:15:42,296::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:42,297::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7db64b1-1858-46d3-a7d5-9276876c2672`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:42,297::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7db64b1-1858-46d3-a7d5-9276876c2672`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:42,297::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:42,297::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:42,298::task::993::Storage.TaskManager.Task::(_decref) Task=`e7db64b1-1858-46d3-a7d5-9276876c2672`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:15:42,308::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:15:42,309::task::595::Storage.TaskManager.Task::(_updateState) Task=`33c2727f-9e49-4f99-8e0b-99bb5978976f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:15:42,309::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:15:42,310::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`960ff040-0330-40e4-b6f3-5023800dae56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:15:42,310::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:15:42,311::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:15:42,311::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`960ff040-0330-40e4-b6f3-5023800dae56`::Granted request >Thread-34::DEBUG::2014-08-11 14:15:42,311::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`33c2727f-9e49-4f99-8e0b-99bb5978976f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:15:42,312::task::993::Storage.TaskManager.Task::(_decref) Task=`33c2727f-9e49-4f99-8e0b-99bb5978976f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:15:42,331::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:42,332::task::1191::Storage.TaskManager.Task::(prepare) Task=`33c2727f-9e49-4f99-8e0b-99bb5978976f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:42,332::task::595::Storage.TaskManager.Task::(_updateState) Task=`33c2727f-9e49-4f99-8e0b-99bb5978976f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:15:42,332::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:15:42,332::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:15:42,333::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:15:42,333::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:15:42,333::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:15:42,333::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:15:42,334::task::993::Storage.TaskManager.Task::(_decref) Task=`33c2727f-9e49-4f99-8e0b-99bb5978976f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:15:43,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:44,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131905 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:46,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:46,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131291 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:47,229::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:47,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:47,303::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:48,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:48,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141145 s, 72.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:50,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:50,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132685 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:52,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:15:52,232::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:52,233::__init__::232::IOProcess::(_processLogs) (4677) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:15:52,243::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:52,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326825 s, 31.3 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:15:52,270::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00753541 s, 60.4 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:15:52,273::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:52,274::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:52,275::__init__::232::IOProcess::(_processLogs) (4678) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:15:52,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:52,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:52,280::__init__::232::IOProcess::(_processLogs) (4679) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:52,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:52,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:52,285::__init__::232::IOProcess::(_processLogs) (4680) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:52,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:15:52,289::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:15:52,291::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:15:52,293::__init__::232::IOProcess::(_processLogs) (4681) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:52,310::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:52,312::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:52,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:15:52,346::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-86::DEBUG::2014-08-11 14:15:52,347::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b91af3c-d121-4899-9628-5e5bc91caefe`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:52,347::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:15:52,360::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:52,360::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b91af3c-d121-4899-9628-5e5bc91caefe`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:15:52,360::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b91af3c-d121-4899-9628-5e5bc91caefe`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:52,360::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:52,361::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:52,361::task::993::Storage.TaskManager.Task::(_decref) Task=`0b91af3c-d121-4899-9628-5e5bc91caefe`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:15:52,372::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-34::DEBUG::2014-08-11 14:15:52,373::task::595::Storage.TaskManager.Task::(_updateState) Task=`358f1b41-e983-49a6-b902-8a7d2f11bc32`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:15:52,373::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:15:52,374::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`630166ae-6fac-41ba-bebc-011c61fc54c0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:15:52,374::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:15:52,374::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:15:52,374::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`630166ae-6fac-41ba-bebc-011c61fc54c0`::Granted request >Thread-34::DEBUG::2014-08-11 14:15:52,375::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`358f1b41-e983-49a6-b902-8a7d2f11bc32`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:15:52,375::task::993::Storage.TaskManager.Task::(_decref) Task=`358f1b41-e983-49a6-b902-8a7d2f11bc32`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:15:52,395::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:52,395::task::1191::Storage.TaskManager.Task::(prepare) Task=`358f1b41-e983-49a6-b902-8a7d2f11bc32`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:15:52,395::task::595::Storage.TaskManager.Task::(_updateState) Task=`358f1b41-e983-49a6-b902-8a7d2f11bc32`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:15:52,396::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:15:52,396::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:15:52,396::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:15:52,397::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:15:52,397::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:15:52,397::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:15:52,397::task::993::Storage.TaskManager.Task::(_decref) Task=`358f1b41-e983-49a6-b902-8a7d2f11bc32`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:15:53,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e5baeb2-51ea-4563-b0a2-05a8a6660bec`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:15:53,234::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:15:53,234::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00753541', 'lastCheck': '0.9', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:53,234::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e5baeb2-51ea-4563-b0a2-05a8a6660bec`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00753541', 'lastCheck': '0.9', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:15:53,235::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e5baeb2-51ea-4563-b0a2-05a8a6660bec`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:15:53,235::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:15:53,235::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:15:53,235::task::993::Storage.TaskManager.Task::(_decref) Task=`8e5baeb2-51ea-4563-b0a2-05a8a6660bec`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:15:53,257::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:54,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:54,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132659 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:15:56,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:56,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132863 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:57,318::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:57,320::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:15:57,403::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:15:58,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:15:58,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133064 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:00,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:00,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133985 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:02,294::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:02,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:02,297::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:02,299::__init__::232::IOProcess::(_processLogs) (4682) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:02,301::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4682) Queuing response >Thread-16::DEBUG::2014-08-11 14:16:02,303::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:16:02,325::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000609138 s, 747 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:02,328::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:02,406::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:02,407::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:02,409::__init__::232::IOProcess::(_processLogs) (4683) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:16:02,412::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:02,413::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:16:02,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf290e2-e0c2-4a81-9d44-9334d423b057`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:02,418::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:16:02,420::__init__::232::IOProcess::(_processLogs) (4683) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:02,421::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:02,424::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:02,425::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:02,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:02,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:02,430::__init__::232::IOProcess::(_processLogs) (4684) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:02,431::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4684) Queuing response >Thread-86::INFO::2014-08-11 14:16:02,434::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:16:02,436::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:16:02,437::task::1191::Storage.TaskManager.Task::(prepare) Task=`daf290e2-e0c2-4a81-9d44-9334d423b057`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:16:02,438::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:16:02,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf290e2-e0c2-4a81-9d44-9334d423b057`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:16:02,440::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:16:02,441::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:16:02,441::__init__::232::IOProcess::(_processLogs) (4685) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:16:02,443::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:16:02,443::__init__::232::IOProcess::(_processLogs) (4685) Queuing response >Thread-86::DEBUG::2014-08-11 14:16:02,445::task::993::Storage.TaskManager.Task::(_decref) Task=`daf290e2-e0c2-4a81-9d44-9334d423b057`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:16:02,446::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:02,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:02,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:02,451::__init__::232::IOProcess::(_processLogs) (4686) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:02,454::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4686) Queuing response >Thread-34::DEBUG::2014-08-11 14:16:02,461::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:16:02,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5f2f950-5aaa-4faf-ab02-bdae8a534d8b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:16:02,466::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:16:02,470::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d7ed977e-9613-44a2-8100-74b6214fce25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:16:02,471::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:16:02,474::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:16:02,475::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d7ed977e-9613-44a2-8100-74b6214fce25`::Granted request >Thread-34::DEBUG::2014-08-11 14:16:02,480::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5f2f950-5aaa-4faf-ab02-bdae8a534d8b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:16:02,481::task::993::Storage.TaskManager.Task::(_decref) Task=`d5f2f950-5aaa-4faf-ab02-bdae8a534d8b`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:16:02,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 14:16:02,544::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:02,546::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5f2f950-5aaa-4faf-ab02-bdae8a534d8b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:02,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5f2f950-5aaa-4faf-ab02-bdae8a534d8b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:16:02,548::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:16:02,550::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:16:02,556::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:16:02,558::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:16:02,560::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:16:02,561::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:16:02,564::task::993::Storage.TaskManager.Task::(_decref) Task=`d5f2f950-5aaa-4faf-ab02-bdae8a534d8b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:16:02,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0364924 s, 28.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:04,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:04,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131614 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:06,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:06,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131748 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:07,435::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:07,436::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:07,505::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:16:08,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e97e270-c7a5-4b2a-8dbb-1947eeb5a2cc`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:08,354::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:16:08,354::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000609138', 'lastCheck': '5.9', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:08,355::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e97e270-c7a5-4b2a-8dbb-1947eeb5a2cc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000609138', 'lastCheck': '5.9', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:08,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e97e270-c7a5-4b2a-8dbb-1947eeb5a2cc`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:08,355::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:08,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:08,355::task::993::Storage.TaskManager.Task::(_decref) Task=`6e97e270-c7a5-4b2a-8dbb-1947eeb5a2cc`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:16:08,385::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:16:08,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`03b45868-9f6b-4558-ace7-ecbfb78860b8`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:16:08,769::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:16:08,770::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:08,771::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:08,771::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:08,771::__init__::232::IOProcess::(_processLogs) (4687) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:16:08,772::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4687) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:08,773::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:08,773::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:16:08,774::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:16:08,775::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:16:08,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`03b45868-9f6b-4558-ace7-ecbfb78860b8`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:16:08,776::__init__::232::IOProcess::(_processLogs) (4688) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:16:08,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`03b45868-9f6b-4558-ace7-ecbfb78860b8`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:16:08,777::__init__::232::IOProcess::(_processLogs) (4688) Queuing response >Thread-55::DEBUG::2014-08-11 14:16:08,778::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:16:08,778::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:16:08,779::task::993::Storage.TaskManager.Task::(_decref) Task=`03b45868-9f6b-4558-ace7-ecbfb78860b8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:16:08,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:08,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132519 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:11,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:11,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131869 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:12,455::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:12,456::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:16:12,457::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:16:12,505::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:12,524::__init__::232::IOProcess::(_processLogs) (4689) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:12,526::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:12,526::__init__::232::IOProcess::(_processLogs) (4689) Queuing response >Thread-16::DEBUG::2014-08-11 14:16:12,528::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000459324 s, 991 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:12,531::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:12,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:12,532::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:12,533::__init__::232::IOProcess::(_processLogs) (4690) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:12,533::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4690) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:12,534::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:12,538::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:12,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:12,540::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:12,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:12,542::__init__::232::IOProcess::(_processLogs) (4691) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:12,544::__init__::232::IOProcess::(_processLogs) (4691) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:12,545::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:12,545::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:12,545::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:12,546::__init__::232::IOProcess::(_processLogs) (4692) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:12,546::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4692) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:12,547::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:12,548::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:12,548::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:12,549::__init__::232::IOProcess::(_processLogs) (4693) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:12,549::__init__::232::IOProcess::(_processLogs) (4693) Queuing response >Thread-86::DEBUG::2014-08-11 14:16:12,580::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:16:12,581::task::595::Storage.TaskManager.Task::(_updateState) Task=`058ed5bf-a10c-4745-b8c6-6ec6ea9f9c39`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:12,581::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:16:12,593::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:12,594::task::1191::Storage.TaskManager.Task::(prepare) Task=`058ed5bf-a10c-4745-b8c6-6ec6ea9f9c39`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:12,594::task::595::Storage.TaskManager.Task::(_updateState) Task=`058ed5bf-a10c-4745-b8c6-6ec6ea9f9c39`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:12,594::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:12,594::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:12,595::task::993::Storage.TaskManager.Task::(_decref) Task=`058ed5bf-a10c-4745-b8c6-6ec6ea9f9c39`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:16:12,606::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:16:12,606::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4a4ee52-8d0b-402d-a62d-6be754d374e8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:16:12,607::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:16:12,607::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a1d4b8c0-7cb1-49de-aff8-449f48033c1b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:16:12,608::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:16:12,608::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:16:12,608::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a1d4b8c0-7cb1-49de-aff8-449f48033c1b`::Granted request >Thread-34::DEBUG::2014-08-11 14:16:12,609::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d4a4ee52-8d0b-402d-a62d-6be754d374e8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:16:12,609::task::993::Storage.TaskManager.Task::(_decref) Task=`d4a4ee52-8d0b-402d-a62d-6be754d374e8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:16:12,629::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:12,629::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4a4ee52-8d0b-402d-a62d-6be754d374e8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:12,629::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4a4ee52-8d0b-402d-a62d-6be754d374e8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:16:12,630::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:16:12,630::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:16:12,630::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:16:12,630::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:16:12,631::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:16:12,631::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:16:12,631::task::993::Storage.TaskManager.Task::(_decref) Task=`d4a4ee52-8d0b-402d-a62d-6be754d374e8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:16:13,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:13,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132135 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:15,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:15,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131839 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:17,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:17,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137423 s, 74.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:17,549::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:17,551::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:17,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:19,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:19,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0206745 s, 49.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:21,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:21,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013288 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:22,550::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:22,551::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:22,606::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:22,607::__init__::232::IOProcess::(_processLogs) (4694) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:22,607::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4694) Queuing response >Thread-16::DEBUG::2014-08-11 14:16:22,609::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:22,626::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:16:22,628::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000443 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:22,631::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:22,631::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:22,633::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:22,634::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:22,636::__init__::232::IOProcess::(_processLogs) (4695) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:22,638::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4695) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:22,638::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:22,641::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:22,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:22,643::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:22,643::__init__::232::IOProcess::(_processLogs) (4696) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:22,644::__init__::232::IOProcess::(_processLogs) (4696) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:22,644::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:22,645::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:22,647::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:16:22,648::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-14::DEBUG::2014-08-11 14:16:22,648::__init__::232::IOProcess::(_processLogs) (4697) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:22,650::__init__::232::IOProcess::(_processLogs) (4697) Queuing response >Thread-86::DEBUG::2014-08-11 14:16:22,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4e77323-6c41-4b63-9143-aa52d9f5714e`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:16:22,653::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-86::INFO::2014-08-11 14:16:22,652::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:16:22,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:22,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:22,656::__init__::232::IOProcess::(_processLogs) (4698) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:22,656::__init__::232::IOProcess::(_processLogs) (4698) Queuing response >Thread-86::INFO::2014-08-11 14:16:22,668::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:22,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4e77323-6c41-4b63-9143-aa52d9f5714e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:22,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4e77323-6c41-4b63-9143-aa52d9f5714e`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:22,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:22,669::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:22,669::task::993::Storage.TaskManager.Task::(_decref) Task=`d4e77323-6c41-4b63-9143-aa52d9f5714e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:16:22,681::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-34::DEBUG::2014-08-11 14:16:22,682::task::595::Storage.TaskManager.Task::(_updateState) Task=`179228cc-3056-40f2-9cf3-2a74b0efae08`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:16:22,682::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:16:22,683::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8e1be655-d1c5-4b6d-be0c-d1aea4f7a596`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:16:22,683::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:16:22,683::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:16:22,684::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8e1be655-d1c5-4b6d-be0c-d1aea4f7a596`::Granted request >Thread-34::DEBUG::2014-08-11 14:16:22,684::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`179228cc-3056-40f2-9cf3-2a74b0efae08`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:16:22,684::task::993::Storage.TaskManager.Task::(_decref) Task=`179228cc-3056-40f2-9cf3-2a74b0efae08`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:16:22,704::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:22,704::task::1191::Storage.TaskManager.Task::(prepare) Task=`179228cc-3056-40f2-9cf3-2a74b0efae08`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085916160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:22,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`179228cc-3056-40f2-9cf3-2a74b0efae08`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:16:22,705::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:16:22,705::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:16:22,706::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:16:22,706::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:16:22,706::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:16:22,706::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:16:22,707::task::993::Storage.TaskManager.Task::(_decref) Task=`179228cc-3056-40f2-9cf3-2a74b0efae08`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:16:23,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:23,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013804 s, 74.2 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:16:23,474::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b929ffd-6aea-4deb-9039-3f26b0b09807`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:23,474::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:16:23,475::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000443', 'lastCheck': '0.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:23,475::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b929ffd-6aea-4deb-9039-3f26b0b09807`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000443', 'lastCheck': '0.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:23,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b929ffd-6aea-4deb-9039-3f26b0b09807`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:23,475::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:23,475::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:23,476::task::993::Storage.TaskManager.Task::(_decref) Task=`0b929ffd-6aea-4deb-9039-3f26b0b09807`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:16:23,495::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:25,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:25,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132254 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:27,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:27,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132278 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:27,648::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:27,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:27,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:29,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:29,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132754 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:31,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:31,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132024 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:32,656::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:32,657::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:32,659::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:32,708::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:32,710::__init__::232::IOProcess::(_processLogs) (4699) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:32,712::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4699) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:32,714::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:16:32,716::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:32,738::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:16:32,741::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-16::DEBUG::2014-08-11 14:16:32,743::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000835837 s, 544 kB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:16:32,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`df387010-a8a4-47b7-aae0-95a56d0f7dd4`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:16:32,750::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-86::INFO::2014-08-11 14:16:32,750::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:16:32,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:32,753::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:32,754::__init__::232::IOProcess::(_processLogs) (4700) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:32,755::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4700) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:32,757::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:32,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:32,759::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:32,760::__init__::232::IOProcess::(_processLogs) (4701) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:32,761::__init__::232::IOProcess::(_processLogs) (4701) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:32,762::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:32,763::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:32,764::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:32,765::__init__::232::IOProcess::(_processLogs) (4702) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:32,765::__init__::232::IOProcess::(_processLogs) (4702) Queuing response >Thread-86::INFO::2014-08-11 14:16:32,767::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:16:32,769::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:16:32,769::task::1191::Storage.TaskManager.Task::(prepare) Task=`df387010-a8a4-47b7-aae0-95a56d0f7dd4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:16:32,770::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:16:32,771::task::595::Storage.TaskManager.Task::(_updateState) Task=`df387010-a8a4-47b7-aae0-95a56d0f7dd4`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:16:32,771::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:16:32,772::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:32,773::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:16:32,773::__init__::232::IOProcess::(_processLogs) (4703) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:16:32,774::task::993::Storage.TaskManager.Task::(_decref) Task=`df387010-a8a4-47b7-aae0-95a56d0f7dd4`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:16:32,775::__init__::232::IOProcess::(_processLogs) (4703) Queuing response >Thread-34::DEBUG::2014-08-11 14:16:32,791::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 14:16:32,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`69a91c45-1b0e-4f82-a0a9-9a83ad27e51d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:16:32,795::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:16:32,798::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`167d4089-c17a-4cbd-8291-028f60a27d04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:16:32,799::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:16:32,801::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:16:32,802::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`167d4089-c17a-4cbd-8291-028f60a27d04`::Granted request >Thread-34::DEBUG::2014-08-11 14:16:32,805::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`69a91c45-1b0e-4f82-a0a9-9a83ad27e51d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:16:32,806::task::993::Storage.TaskManager.Task::(_decref) Task=`69a91c45-1b0e-4f82-a0a9-9a83ad27e51d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:16:32,860::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085850624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:32,861::task::1191::Storage.TaskManager.Task::(prepare) Task=`69a91c45-1b0e-4f82-a0a9-9a83ad27e51d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085850624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:32,862::task::595::Storage.TaskManager.Task::(_updateState) Task=`69a91c45-1b0e-4f82-a0a9-9a83ad27e51d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:16:32,863::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:16:32,864::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:16:32,866::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:16:32,869::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:16:32,870::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:16:32,872::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:16:32,875::task::993::Storage.TaskManager.Task::(_decref) Task=`69a91c45-1b0e-4f82-a0a9-9a83ad27e51d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:16:33,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:33,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132766 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:35,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:35,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132281 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:37,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:37,760::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:37,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132281 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:37,811::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:37,815::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:16:38,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8aafbea-467b-4103-b95d-6cb18791f126`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:38,598::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:16:38,599::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000835837', 'lastCheck': '5.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:38,599::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8aafbea-467b-4103-b95d-6cb18791f126`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000835837', 'lastCheck': '5.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:38,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8aafbea-467b-4103-b95d-6cb18791f126`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:38,599::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:38,600::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:38,600::task::993::Storage.TaskManager.Task::(_decref) Task=`a8aafbea-467b-4103-b95d-6cb18791f126`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:16:38,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:39,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:39,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133815 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:41,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:41,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132049 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:42,773::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:42,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:42,807::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:16:42,808::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:16:42,809::__init__::232::IOProcess::(_processLogs) (4704) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:42,826::__init__::232::IOProcess::(_processLogs) (4704) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:42,828::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:42,829::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:42,831::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:16:42,833::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042428 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:42,834::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:42,835::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:42,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:42,835::__init__::232::IOProcess::(_processLogs) (4705) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:42,836::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4705) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:42,837::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:42,838::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:42,838::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:42,839::__init__::232::IOProcess::(_processLogs) (4706) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:42,839::__init__::232::IOProcess::(_processLogs) (4706) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:42,839::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:42,840::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:42,840::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:42,840::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4707) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:42,841::__init__::232::IOProcess::(_processLogs) (4707) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:42,841::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:42,842::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:42,842::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:42,843::__init__::232::IOProcess::(_processLogs) (4708) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:42,843::__init__::232::IOProcess::(_processLogs) (4708) Queuing response >Thread-86::DEBUG::2014-08-11 14:16:42,895::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:16:42,895::task::595::Storage.TaskManager.Task::(_updateState) Task=`e30a6b70-8d28-4bda-a435-ecb2b2833403`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:42,896::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:16:42,909::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:42,909::task::1191::Storage.TaskManager.Task::(prepare) Task=`e30a6b70-8d28-4bda-a435-ecb2b2833403`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:42,910::task::595::Storage.TaskManager.Task::(_updateState) Task=`e30a6b70-8d28-4bda-a435-ecb2b2833403`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:42,910::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:42,910::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:42,910::task::993::Storage.TaskManager.Task::(_decref) Task=`e30a6b70-8d28-4bda-a435-ecb2b2833403`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:16:42,920::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:16:42,921::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dcc1b27-2c89-4735-bc29-c038a6838bba`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:16:42,921::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:16:42,922::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7c444d68-22a8-4e29-96d3-3dc253bd2104`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:16:42,922::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:16:42,923::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:16:42,923::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7c444d68-22a8-4e29-96d3-3dc253bd2104`::Granted request >Thread-34::DEBUG::2014-08-11 14:16:42,923::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0dcc1b27-2c89-4735-bc29-c038a6838bba`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:16:42,924::task::993::Storage.TaskManager.Task::(_decref) Task=`0dcc1b27-2c89-4735-bc29-c038a6838bba`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:16:42,943::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:42,944::task::1191::Storage.TaskManager.Task::(prepare) Task=`0dcc1b27-2c89-4735-bc29-c038a6838bba`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:42,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dcc1b27-2c89-4735-bc29-c038a6838bba`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:16:42,944::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:16:42,944::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:16:42,945::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:16:42,945::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:16:42,945::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:16:42,945::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:16:42,946::task::993::Storage.TaskManager.Task::(_decref) Task=`0dcc1b27-2c89-4735-bc29-c038a6838bba`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:16:43,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:43,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131808 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:46,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:46,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131924 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:47,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:47,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:47,842::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:48,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:48,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132296 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:50,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:50,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133823 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:52,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:52,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131248 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:52,844::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:52,844::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:52,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:16:52,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:52,909::__init__::232::IOProcess::(_processLogs) (4709) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:52,910::__init__::232::IOProcess::(_processLogs) (4709) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:52,911::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:16:52,913::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:52,932::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:16:52,933::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00222368 s, 205 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:16:52,934::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:52,935::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:52,935::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:52,935::__init__::232::IOProcess::(_processLogs) (4710) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:16:52,936::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4710) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:52,937::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:52,938::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:52,939::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:52,939::__init__::232::IOProcess::(_processLogs) (4711) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:52,939::__init__::232::IOProcess::(_processLogs) (4711) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:52,939::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:52,940::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:52,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:52,941::__init__::232::IOProcess::(_processLogs) (4712) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:52,941::__init__::232::IOProcess::(_processLogs) (4712) Queuing response >Thread-14::DEBUG::2014-08-11 14:16:52,942::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:16:52,942::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:16:52,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:16:52,944::__init__::232::IOProcess::(_processLogs) (4713) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:16:52,944::__init__::232::IOProcess::(_processLogs) (4713) Queuing response >Thread-86::DEBUG::2014-08-11 14:16:52,958::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:16:52,959::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c366cfa-ca94-4a2c-a176-6d69203c0abc`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:52,959::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:16:52,972::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:52,972::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c366cfa-ca94-4a2c-a176-6d69203c0abc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:16:52,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c366cfa-ca94-4a2c-a176-6d69203c0abc`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:52,972::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:52,973::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:52,973::task::993::Storage.TaskManager.Task::(_decref) Task=`1c366cfa-ca94-4a2c-a176-6d69203c0abc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:16:52,983::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:16:52,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`23ab4840-f69f-46bb-addd-4ca83ace0c23`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:16:52,984::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:16:52,984::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b25f472f-70af-498b-ad66-71d16420337d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:16:52,985::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:16:52,985::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:16:52,985::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b25f472f-70af-498b-ad66-71d16420337d`::Granted request >Thread-34::DEBUG::2014-08-11 14:16:52,986::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`23ab4840-f69f-46bb-addd-4ca83ace0c23`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:16:52,986::task::993::Storage.TaskManager.Task::(_decref) Task=`23ab4840-f69f-46bb-addd-4ca83ace0c23`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:16:53,006::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:53,006::task::1191::Storage.TaskManager.Task::(prepare) Task=`23ab4840-f69f-46bb-addd-4ca83ace0c23`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:16:53,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`23ab4840-f69f-46bb-addd-4ca83ace0c23`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:16:53,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:16:53,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:16:53,007::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:16:53,008::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:16:53,008::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:16:53,008::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:16:53,008::task::993::Storage.TaskManager.Task::(_decref) Task=`23ab4840-f69f-46bb-addd-4ca83ace0c23`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:16:53,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b102c46-b691-400a-8be6-91caba9033eb`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:16:53,706::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:16:53,707::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00222368', 'lastCheck': '0.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:53,707::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b102c46-b691-400a-8be6-91caba9033eb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00222368', 'lastCheck': '0.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:16:53,707::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b102c46-b691-400a-8be6-91caba9033eb`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:16:53,707::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:16:53,708::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:16:53,708::task::993::Storage.TaskManager.Task::(_decref) Task=`0b102c46-b691-400a-8be6-91caba9033eb`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:16:53,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:54,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:54,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131103 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:16:56,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:56,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132742 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:57,939::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:57,940::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:16:58,007::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:16:58,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:16:58,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133961 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:00,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:00,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0230157 s, 44.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:02,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:02,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323151 s, 31.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:17:02,944::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:02,945::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:03,007::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:17:03,008::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:17:03,008::__init__::232::IOProcess::(_processLogs) (4714) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:03,028::__init__::232::IOProcess::(_processLogs) (4714) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:03,032::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:17:03,032::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000731713 s, 622 kB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:17:03,035::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:17:03,036::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:17:03,040::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9195542-653e-4b9c-9099-1b7d477ed573`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:03,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:03,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::INFO::2014-08-11 14:17:03,042::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:17:03,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:03,045::__init__::232::IOProcess::(_processLogs) (4715) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:03,046::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|(4715) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:03,047::__init__::232::IOProcess::(_processLogs) (4681) Got request for method 'access'DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:03,049::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:03,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:03,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:03,052::__init__::232::IOProcess::(_processLogs) (4716) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:03,053::__init__::232::IOProcess::(_processLogs) (4716) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:03,055::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:03,055::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:03,056::__init__::232::IOProcess::(_processLogs) (4717) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:03,057::__init__::232::IOProcess::(_processLogs) (4717) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:03,059::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-86::INFO::2014-08-11 14:17:03,059::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:03,061::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:17:03,061::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9195542-653e-4b9c-9099-1b7d477ed573`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:03,063::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:17:03,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9195542-653e-4b9c-9099-1b7d477ed573`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:17:03,064::__init__::232::IOProcess::(_processLogs) (4718) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:17:03,064::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:17:03,065::__init__::232::IOProcess::(_processLogs) (4718) Queuing response >Thread-86::DEBUG::2014-08-11 14:17:03,066::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:03,067::task::993::Storage.TaskManager.Task::(_decref) Task=`c9195542-653e-4b9c-9099-1b7d477ed573`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:17:03,077::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:17:03,077::task::595::Storage.TaskManager.Task::(_updateState) Task=`531f77db-c35b-4c16-9d45-f353c080355e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:17:03,078::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:17:03,080::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`597b2657-faa5-4145-964b-5a7423bc405b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:17:03,081::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:17:03,081::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:17:03,081::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`597b2657-faa5-4145-964b-5a7423bc405b`::Granted request >Thread-34::DEBUG::2014-08-11 14:17:03,082::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`531f77db-c35b-4c16-9d45-f353c080355e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:17:03,082::task::993::Storage.TaskManager.Task::(_decref) Task=`531f77db-c35b-4c16-9d45-f353c080355e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:17:03,102::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:03,102::task::1191::Storage.TaskManager.Task::(prepare) Task=`531f77db-c35b-4c16-9d45-f353c080355e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:03,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`531f77db-c35b-4c16-9d45-f353c080355e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:17:03,103::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:17:03,103::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:17:03,103::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:17:03,104::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:17:03,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:17:03,104::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:17:03,104::task::993::Storage.TaskManager.Task::(_decref) Task=`531f77db-c35b-4c16-9d45-f353c080355e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:04,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:04,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131508 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:06,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:06,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013158 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:08,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:08,066::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:08,067::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:17:08,838::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e199fcf-baa9-4a9f-ac71-9aa19c77a1b1`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:17:08,839::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:17:08,839::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000731713', 'lastCheck': '5.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:08,839::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e199fcf-baa9-4a9f-ac71-9aa19c77a1b1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000731713', 'lastCheck': '5.8', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:08,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e199fcf-baa9-4a9f-ac71-9aa19c77a1b1`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:17:08,840::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:17:08,840::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:08,840::task::993::Storage.TaskManager.Task::(_decref) Task=`2e199fcf-baa9-4a9f-ac71-9aa19c77a1b1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:08,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-86::DEBUG::2014-08-11 14:17:08,867::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:17:08,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132532 s, 77.3 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:17:08,883::task::595::Storage.TaskManager.Task::(_updateState) Task=`a778e5a7-e35c-48c6-a8f3-55cc22c1a0ab`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:17:08,885::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:17:08,888::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:08,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:08,890::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:08,891::__init__::232::IOProcess::(_processLogs) (4719) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:17:08,893::__init__::232::IOProcess::(_processLogs) DEBUG|(4719) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:08,895::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:08,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:08,897::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:08,898::__init__::232::IOProcess::(_processLogs) (4720) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:17:08,899::__init__::232::IOProcess::(_processLogs) (4720) Queuing response >Thread-55::INFO::2014-08-11 14:17:08,901::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:17:08,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`a778e5a7-e35c-48c6-a8f3-55cc22c1a0ab`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:17:08,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`a778e5a7-e35c-48c6-a8f3-55cc22c1a0ab`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:17:08,904::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:17:08,905::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:17:08,906::task::993::Storage.TaskManager.Task::(_decref) Task=`a778e5a7-e35c-48c6-a8f3-55cc22c1a0ab`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:10,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:10,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131538 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:12,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:13,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132603 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:17:13,063::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:13,064::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:13,064::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:13,064::__init__::232::IOProcess::(_processLogs) (4721) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:13,065::__init__::232::IOProcess::(_processLogs) DEBUG|(4721) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:13,074::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:17:13,108::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:17:13,130::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-16::DEBUG::2014-08-11 14:17:13,131::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000707619 s, 643 kB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:17:13,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c807bfa-269d-4ced-bc84-085d95cf4d18`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:13,134::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:13,135::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-86::INFO::2014-08-11 14:17:13,136::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:17:13,137::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:13,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:13,140::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:13,141::__init__::232::IOProcess::(_processLogs) (4722) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:13,143::__init__::232::IOProcess::(_processLogs) DEBUG|(4722) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:13,146::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:13,147::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:13,149::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:13,150::__init__::232::IOProcess::(_processLogs) (4723) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:17:13,151::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:13,151::__init__::232::IOProcess::(_processLogs) (4723) Queuing response >Thread-86::DEBUG::2014-08-11 14:17:13,152::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c807bfa-269d-4ced-bc84-085d95cf4d18`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:13,153::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:17:13,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c807bfa-269d-4ced-bc84-085d95cf4d18`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:17:13,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:17:13,154::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:17:13,156::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:17:13,156::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:17:13,157::__init__::232::IOProcess::(_processLogs) (4724) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:17:13,157::task::993::Storage.TaskManager.Task::(_decref) Task=`0c807bfa-269d-4ced-bc84-085d95cf4d18`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:17:13,159::__init__::232::IOProcess::(_processLogs) DEBUG|(4724) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:13,161::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:13,161::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:13,162::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:13,163::__init__::232::IOProcess::(_processLogs) (4725) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:13,163::__init__::232::IOProcess::(_processLogs) (4725) Queuing response >Thread-34::DEBUG::2014-08-11 14:17:13,170::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-34::DEBUG::2014-08-11 14:17:13,171::task::595::Storage.TaskManager.Task::(_updateState) Task=`9be3a13d-7368-4f14-8715-c76936f359ab`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:17:13,171::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:17:13,172::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8f9d9cf3-782e-4330-b22b-f289d8be09aa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:17:13,172::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:17:13,172::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:17:13,173::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8f9d9cf3-782e-4330-b22b-f289d8be09aa`::Granted request >Thread-34::DEBUG::2014-08-11 14:17:13,173::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9be3a13d-7368-4f14-8715-c76936f359ab`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:17:13,173::task::993::Storage.TaskManager.Task::(_decref) Task=`9be3a13d-7368-4f14-8715-c76936f359ab`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:17:13,193::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:13,193::task::1191::Storage.TaskManager.Task::(prepare) Task=`9be3a13d-7368-4f14-8715-c76936f359ab`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:13,194::task::595::Storage.TaskManager.Task::(_updateState) Task=`9be3a13d-7368-4f14-8715-c76936f359ab`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:17:13,194::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:17:13,194::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:17:13,194::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:17:13,195::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:17:13,195::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:17:13,195::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:17:13,196::task::993::Storage.TaskManager.Task::(_decref) Task=`9be3a13d-7368-4f14-8715-c76936f359ab`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:15,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:15,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013246 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:17,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:17,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130507 s, 78.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:18,151::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:18,153::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:18,207::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:17:19,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:19,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129698 s, 79.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:21,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:21,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176559 s, 58.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:17:23,163::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:17:23,165::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,207::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,209::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:17:23,210::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:17:23,211::__init__::232::IOProcess::(_processLogs) (4726) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:17:23,214::__init__::232::IOProcess::(_processLogs) (4726) Queuing response >Thread-86::DEBUG::2014-08-11 14:17:23,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ac0802c-2f66-4cd9-a197-606a6ee3abbb`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:23,218::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:17:23,221::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:23,226::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:23,233::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:17:23,240::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:17:23,241::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,241::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,241::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:23,242::__init__::232::IOProcess::(_processLogs) DEBUG|(4727) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 14:17:23,242::__init__::232::IOProcess::(_processLogs) DEBUG|(4727) Queuing response >Thread-16::DEBUG::2014-08-11 14:17:23,243::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:17:23,244::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,244::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,244::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:23,244::__init__::232::IOProcess::(_processLogs) (4728) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:17:23,245::__init__::232::IOProcess::(_processLogs) DEBUG|(4728) Queuing response >Thread-16::DEBUG::2014-08-11 14:17:23,245::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:17:23,247::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,248::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:23,248::__init__::232::IOProcess::(_processLogs) (4729) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:23,249::__init__::232::IOProcess::(_processLogs) (4729) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:23,249::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,249::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,250::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:23,250::__init__::232::IOProcess::(_processLogs) (4730) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:17:23,250::__init__::232::IOProcess::(_processLogs) DEBUG|(4730) Queuing response >Thread-16::DEBUG::2014-08-11 14:17:23,251::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 14:17:23,252::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:17:23,252::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:17:23,253::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,253::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:23,254::__init__::232::IOProcess::(_processLogs) DEBUG|(4731) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:23,254::__init__::232::IOProcess::(_processLogs) (4731) Queuing response >Thread-16::DEBUG::2014-08-11 14:17:23,256::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::INFO::2014-08-11 14:17:23,277::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Dummy-33::DEBUG::2014-08-11 14:17:23,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-86::DEBUG::2014-08-11 14:17:23,279::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ac0802c-2f66-4cd9-a197-606a6ee3abbb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:17:23,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ac0802c-2f66-4cd9-a197-606a6ee3abbb`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:17:23,298::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-16::DEBUG::2014-08-11 14:17:23,299::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000969402 s, 469 kB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:17:23,299::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:23,301::task::993::Storage.TaskManager.Task::(_decref) Task=`7ac0802c-2f66-4cd9-a197-606a6ee3abbb`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:17:23,303::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,304::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,305::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:23,305::__init__::232::IOProcess::(_processLogs) DEBUG|(4732) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:23,311::__init__::232::IOProcess::(_processLogs) DEBUG|(4732) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:23,315::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::DEBUG::2014-08-11 14:17:23,317::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:17:23,319::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-34::DEBUG::2014-08-11 14:17:23,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`d08ff205-1ea0-4483-be0c-1f5b980245a6`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:17:23,321::__init__::232::IOProcess::(_processLogs) (4733) Got request for method 'access' >Thread-34::INFO::2014-08-11 14:17:23,321::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:17:23,322::__init__::232::IOProcess::(_processLogs) (4733) Queuing response >Thread-34::DEBUG::2014-08-11 14:17:23,326::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a308ca50-a891-4680-92e8-b6ad0d257b45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:17:23,328::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-14::DEBUG::2014-08-11 14:17:23,328::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::DEBUG::2014-08-11 14:17:23,331::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-14::DEBUG::2014-08-11 14:17:23,332::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-34::DEBUG::2014-08-11 14:17:23,333::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a308ca50-a891-4680-92e8-b6ad0d257b45`::Granted request >Thread-14::DEBUG::2014-08-11 14:17:23,334::__init__::232::IOProcess::(_processLogs) (4734) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:23,336::__init__::232::IOProcess::(_processLogs) (4734) Queuing response >Thread-34::DEBUG::2014-08-11 14:17:23,336::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d08ff205-1ea0-4483-be0c-1f5b980245a6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:17:23,338::task::993::Storage.TaskManager.Task::(_decref) Task=`d08ff205-1ea0-4483-be0c-1f5b980245a6`::ref 1 aborting False >Thread-14::DEBUG::2014-08-11 14:17:23,339::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:23,341::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:23,343::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:23,345::__init__::232::IOProcess::(_processLogs) (4735) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:23,347::__init__::232::IOProcess::(_processLogs) (4735) Queuing response >Dummy-33::DEBUG::2014-08-11 14:17:23,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0511932 s, 20.0 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 14:17:23,391::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:23,392::task::1191::Storage.TaskManager.Task::(prepare) Task=`d08ff205-1ea0-4483-be0c-1f5b980245a6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:23,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`d08ff205-1ea0-4483-be0c-1f5b980245a6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:17:23,394::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:17:23,395::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:17:23,397::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:17:23,398::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:17:23,399::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:17:23,400::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:17:23,402::task::993::Storage.TaskManager.Task::(_decref) Task=`d08ff205-1ea0-4483-be0c-1f5b980245a6`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:17:23,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b0d5fbb-4582-4e16-8024-8b29ebdb83dc`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:17:23,958::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:17:23,959::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000969402', 'lastCheck': '0.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:23,959::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b0d5fbb-4582-4e16-8024-8b29ebdb83dc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000969402', 'lastCheck': '0.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:23,959::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b0d5fbb-4582-4e16-8024-8b29ebdb83dc`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:17:23,959::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:17:23,959::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:23,960::task::993::Storage.TaskManager.Task::(_decref) Task=`6b0d5fbb-4582-4e16-8024-8b29ebdb83dc`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:17:23,980::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:17:25,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:25,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131457 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:27,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:27,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132291 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:28,242::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:28,244::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:28,307::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:17:29,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:29,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132507 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:31,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:31,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135396 s, 75.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:33,314::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:33,315::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:33,352::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:33,409::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:17:33,410::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:17:33,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:17:33,431::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:33,431::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:33,433::__init__::232::IOProcess::(_processLogs) (4736) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:17:33,434::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00127716 s, 356 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:17:33,437::__init__::232::IOProcess::(_processLogs) (4736) Queuing response >Thread-86::DEBUG::2014-08-11 14:17:33,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`82055a23-fbf4-4641-b529-00eaba821668`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:17:33,441::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:17:33,441::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:33,442::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:33,443::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:33,443::__init__::232::IOProcess::(_processLogs) (4737) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:33,444::__init__::232::IOProcess::(_processLogs) DEBUG|(4737) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:33,446::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:33,447::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:33,447::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:33,448::__init__::232::IOProcess::(_processLogs) (4738) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:33,449::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4738) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:33,450::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:33,450::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:33,451::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:33,451::__init__::232::IOProcess::(_processLogs) (4739) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:33,452::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4739) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:33,453::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:33,453::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:33,454::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:33,454::__init__::232::IOProcess::(_processLogs) (4740) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:33,455::__init__::232::IOProcess::(_processLogs) (4740) Queuing response >Thread-86::INFO::2014-08-11 14:17:33,456::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:17:33,456::task::1191::Storage.TaskManager.Task::(prepare) Task=`82055a23-fbf4-4641-b529-00eaba821668`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:17:33,457::task::595::Storage.TaskManager.Task::(_updateState) Task=`82055a23-fbf4-4641-b529-00eaba821668`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:17:33,457::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:17:33,457::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:33,457::task::993::Storage.TaskManager.Task::(_decref) Task=`82055a23-fbf4-4641-b529-00eaba821668`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:17:33,467::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:17:33,468::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5f6be88-1129-4f66-9c94-c363031d738c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:17:33,468::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:17:33,469::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`880c09ca-515a-4849-be50-14ba81d5c4a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:17:33,469::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:17:33,469::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:17:33,470::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`880c09ca-515a-4849-be50-14ba81d5c4a0`::Granted request >Thread-34::DEBUG::2014-08-11 14:17:33,470::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a5f6be88-1129-4f66-9c94-c363031d738c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:17:33,470::task::993::Storage.TaskManager.Task::(_decref) Task=`a5f6be88-1129-4f66-9c94-c363031d738c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:17:33,490::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:33,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`a5f6be88-1129-4f66-9c94-c363031d738c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:33,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5f6be88-1129-4f66-9c94-c363031d738c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:17:33,491::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:17:33,491::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:17:33,491::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:17:33,492::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:17:33,492::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:17:33,492::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:17:33,492::task::993::Storage.TaskManager.Task::(_decref) Task=`a5f6be88-1129-4f66-9c94-c363031d738c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:33,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:33,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132687 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:35,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:35,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131824 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:37,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:37,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132389 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:38,446::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:38,448::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:38,507::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:17:39,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1340f83-b3dc-4070-afb9-5281e9e0c592`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:17:39,071::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:17:39,071::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00127716', 'lastCheck': '5.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:39,071::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1340f83-b3dc-4070-afb9-5281e9e0c592`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00127716', 'lastCheck': '5.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:39,071::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1340f83-b3dc-4070-afb9-5281e9e0c592`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:17:39,072::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:17:39,072::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:39,072::task::993::Storage.TaskManager.Task::(_decref) Task=`e1340f83-b3dc-4070-afb9-5281e9e0c592`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:17:39,091::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:17:39,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:39,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131344 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:41,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:41,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131914 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:17:43,457::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:43,457::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:43,458::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:43,458::__init__::232::IOProcess::(_processLogs) (4741) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:43,459::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4741) Queuing response >Thread-16::DEBUG::2014-08-11 14:17:43,460::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:17:43,505::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:17:43,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`beeae78f-1a80-450d-a6f0-553c71a37c62`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:43,528::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:17:43,528::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 14:17:43,530::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000520463 s, 874 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:17:43,533::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:43,534::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:43,534::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:43,535::__init__::232::IOProcess::(_processLogs) (4742) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:17:43,535::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4742) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:43,536::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:43,541::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:43,542::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:43,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:43,544::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::INFO::2014-08-11 14:17:43,545::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:43,545::__init__::232::IOProcess::(_processLogs) (4743) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:17:43,546::task::1191::Storage.TaskManager.Task::(prepare) Task=`beeae78f-1a80-450d-a6f0-553c71a37c62`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:43,547::__init__::232::IOProcess::(_processLogs) (4743) Queuing response >Thread-86::DEBUG::2014-08-11 14:17:43,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`beeae78f-1a80-450d-a6f0-553c71a37c62`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:17:43,549::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:17:43,549::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:17:43,549::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:43,551::task::993::Storage.TaskManager.Task::(_decref) Task=`beeae78f-1a80-450d-a6f0-553c71a37c62`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:17:43,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:43,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:43,554::__init__::232::IOProcess::(_processLogs) (4744) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:43,554::__init__::232::IOProcess::(_processLogs) (4744) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:43,555::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:43,556::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:43,556::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:43,556::__init__::232::IOProcess::(_processLogs) (4745) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:43,557::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4745) Queuing response >Thread-34::DEBUG::2014-08-11 14:17:43,564::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:17:43,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ec1c779-9e61-4ea1-80c4-9b8b66d677c2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:17:43,565::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:17:43,566::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5146d921-a5f0-45ed-ae97-e023c1be8a53`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:17:43,566::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:17:43,566::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:17:43,566::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5146d921-a5f0-45ed-ae97-e023c1be8a53`::Granted request >Thread-34::DEBUG::2014-08-11 14:17:43,567::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3ec1c779-9e61-4ea1-80c4-9b8b66d677c2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:17:43,567::task::993::Storage.TaskManager.Task::(_decref) Task=`3ec1c779-9e61-4ea1-80c4-9b8b66d677c2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:17:43,587::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:43,587::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ec1c779-9e61-4ea1-80c4-9b8b66d677c2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:43,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ec1c779-9e61-4ea1-80c4-9b8b66d677c2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:17:43,588::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:17:43,588::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:17:43,588::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:17:43,589::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:17:43,589::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:17:43,589::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:17:43,589::task::993::Storage.TaskManager.Task::(_decref) Task=`3ec1c779-9e61-4ea1-80c4-9b8b66d677c2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:43,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:43,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132756 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:45,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:45,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132593 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:48,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:48,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131886 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:48,548::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:48,550::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:48,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:17:50,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:50,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132822 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:52,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:52,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131513 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:17:53,558::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:53,559::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:17:53,601::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:17:53,609::task::595::Storage.TaskManager.Task::(_updateState) Task=`93090d07-15a8-4e5f-a4ab-4b7dc82a9469`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:17:53,608::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:17:53,610::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::INFO::2014-08-11 14:17:53,611::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:17:53,630::__init__::232::IOProcess::(_processLogs) (4746) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:53,632::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:53,634::__init__::232::IOProcess::(_processLogs) (4746) Queuing response >Thread-16::DEBUG::2014-08-11 14:17:53,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000680953 s, 668 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:53,637::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:53,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:17:53,643::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-86::INFO::2014-08-11 14:17:53,644::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:53,645::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:17:53,646::task::1191::Storage.TaskManager.Task::(prepare) Task=`93090d07-15a8-4e5f-a4ab-4b7dc82a9469`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:17:53,647::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:17:53,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`93090d07-15a8-4e5f-a4ab-4b7dc82a9469`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:17:53,648::__init__::232::IOProcess::(_processLogs) (4747) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:17:53,648::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:17:53,650::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4747) Queuing response >Thread-86::DEBUG::2014-08-11 14:17:53,649::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:53,652::task::993::Storage.TaskManager.Task::(_decref) Task=`93090d07-15a8-4e5f-a4ab-4b7dc82a9469`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:17:53,652::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:53,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:53,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:53,656::__init__::232::IOProcess::(_processLogs) (4748) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:53,656::__init__::232::IOProcess::(_processLogs) (4748) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:53,657::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:53,657::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:53,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:53,658::__init__::232::IOProcess::(_processLogs) (4749) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:53,659::__init__::232::IOProcess::(_processLogs) (4749) Queuing response >Thread-14::DEBUG::2014-08-11 14:17:53,659::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:17:53,660::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:17:53,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:17:53,661::__init__::232::IOProcess::(_processLogs) (4750) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:17:53,661::__init__::232::IOProcess::(_processLogs) (4750) Queuing response >Thread-34::DEBUG::2014-08-11 14:17:53,666::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:17:53,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`57e2da33-bef4-4739-80e2-eb18656f685e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:17:53,667::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:17:53,668::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c29d24c7-034f-40c7-9220-b09e6ad5b366`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:17:53,668::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:17:53,668::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:17:53,668::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c29d24c7-034f-40c7-9220-b09e6ad5b366`::Granted request >Thread-34::DEBUG::2014-08-11 14:17:53,669::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`57e2da33-bef4-4739-80e2-eb18656f685e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:17:53,669::task::993::Storage.TaskManager.Task::(_decref) Task=`57e2da33-bef4-4739-80e2-eb18656f685e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:17:53,689::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:53,689::task::1191::Storage.TaskManager.Task::(prepare) Task=`57e2da33-bef4-4739-80e2-eb18656f685e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085785088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:17:53,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`57e2da33-bef4-4739-80e2-eb18656f685e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:17:53,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:17:53,690::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:17:53,690::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:17:53,691::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:17:53,691::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:17:53,691::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:17:53,692::task::993::Storage.TaskManager.Task::(_decref) Task=`57e2da33-bef4-4739-80e2-eb18656f685e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:54,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-86::DEBUG::2014-08-11 14:17:54,198::task::595::Storage.TaskManager.Task::(_updateState) Task=`d944c0c9-36cd-4e94-929d-ab1ed6dcdded`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:17:54,199::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:17:54,200::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000680953', 'lastCheck': '0.5', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:54,200::task::1191::Storage.TaskManager.Task::(prepare) Task=`d944c0c9-36cd-4e94-929d-ab1ed6dcdded`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000680953', 'lastCheck': '0.5', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:17:54,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`d944c0c9-36cd-4e94-929d-ab1ed6dcdded`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:17:54,201::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:17:54,202::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:17:54,203::task::993::Storage.TaskManager.Task::(_decref) Task=`d944c0c9-36cd-4e94-929d-ab1ed6dcdded`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:17:54,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249918 s, 41.0 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:17:54,233::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:17:56,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:56,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132599 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:17:58,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:17:58,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131994 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:58,651::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:58,652::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:17:58,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:00,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:00,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132926 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:02,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:02,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234238 s, 43.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:03,661::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:03,662::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:18:03,703::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:18:03,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`04dc125b-cfef-42b0-93a0-e40695bc9655`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:18:03,710::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:18:03,710::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::INFO::2014-08-11 14:18:03,712::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:18:03,712::__init__::232::IOProcess::(_processLogs) (4751) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:18:03,731::__init__::232::IOProcess::(_processLogs) (4751) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:03,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:18:03,735::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000621213 s, 732 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:03,737::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:03,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:03,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:03,739::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4752) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:03,742::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:18:03,744::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4752) Queuing response >Thread-86::INFO::2014-08-11 14:18:03,745::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:03,747::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:18:03,748::task::1191::Storage.TaskManager.Task::(prepare) Task=`04dc125b-cfef-42b0-93a0-e40695bc9655`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:03,749::task::595::Storage.TaskManager.Task::(_updateState) Task=`04dc125b-cfef-42b0-93a0-e40695bc9655`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:03,750::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:18:03,750::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:18:03,751::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:18:03,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:18:03,753::task::993::Storage.TaskManager.Task::(_decref) Task=`04dc125b-cfef-42b0-93a0-e40695bc9655`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:18:03,754::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:03,755::__init__::232::IOProcess::(_processLogs) (4753) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:03,756::__init__::232::IOProcess::(_processLogs) (4753) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:03,757::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:03,757::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:03,758::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:03,758::__init__::232::IOProcess::(_processLogs) (4754) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:03,759::__init__::232::IOProcess::(_processLogs) (4754) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:03,759::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:03,759::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:03,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:03,760::__init__::232::IOProcess::(_processLogs) (4755) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:03,760::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4755) Queuing response >Thread-34::DEBUG::2014-08-11 14:18:03,765::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:18:03,765::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2fffc30-e2e6-4b50-892f-c36e51228d54`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:18:03,766::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:18:03,766::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f6c998fc-b988-430d-a25d-c686b2bd716c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:18:03,767::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:18:03,767::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:18:03,767::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f6c998fc-b988-430d-a25d-c686b2bd716c`::Granted request >Thread-34::DEBUG::2014-08-11 14:18:03,768::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e2fffc30-e2e6-4b50-892f-c36e51228d54`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:18:03,768::task::993::Storage.TaskManager.Task::(_decref) Task=`e2fffc30-e2e6-4b50-892f-c36e51228d54`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:18:03,788::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085719552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:03,788::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2fffc30-e2e6-4b50-892f-c36e51228d54`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085719552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:03,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2fffc30-e2e6-4b50-892f-c36e51228d54`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:18:03,789::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:18:03,789::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:18:03,789::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:18:03,790::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:18:03,790::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:18:03,790::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:18:03,790::task::993::Storage.TaskManager.Task::(_decref) Task=`e2fffc30-e2e6-4b50-892f-c36e51228d54`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:18:04,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:04,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133579 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:06,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:06,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131253 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:08,760::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:08,762::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:08,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:08,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:08,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133191 s, 76.9 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:18:09,014::task::595::Storage.TaskManager.Task::(_updateState) Task=`807111d2-6ec8-480d-993a-24c0619e9679`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:18:09,015::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:18:09,016::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:09,016::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:09,016::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:09,017::__init__::232::IOProcess::(_processLogs) (4756) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:18:09,017::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4756) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:09,018::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:09,018::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:09,019::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::INFO::2014-08-11 14:18:09,019::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:18:09,019::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(4757) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:18:09,021::task::1191::Storage.TaskManager.Task::(prepare) Task=`807111d2-6ec8-480d-993a-24c0619e9679`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:18:09,021::__init__::232::IOProcess::(_processLogs) (4757) Queuing response >Thread-55::DEBUG::2014-08-11 14:18:09,022::task::595::Storage.TaskManager.Task::(_updateState) Task=`807111d2-6ec8-480d-993a-24c0619e9679`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:18:09,022::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:18:09,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:18:09,023::task::993::Storage.TaskManager.Task::(_decref) Task=`807111d2-6ec8-480d-993a-24c0619e9679`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:18:09,341::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfaff58c-2204-4767-9f34-c16c0208b58f`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:18:09,342::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:18:09,342::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000621213', 'lastCheck': '5.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:09,342::task::1191::Storage.TaskManager.Task::(prepare) Task=`cfaff58c-2204-4767-9f34-c16c0208b58f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000621213', 'lastCheck': '5.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:09,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfaff58c-2204-4767-9f34-c16c0208b58f`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:09,343::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:18:09,343::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:09,343::task::993::Storage.TaskManager.Task::(_decref) Task=`cfaff58c-2204-4767-9f34-c16c0208b58f`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:18:09,362::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:10,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:10,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132061 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:12,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:12,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131239 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:13,763::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:13,764::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:18:13,764::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:18:13,804::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:18:13,808::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:13,826::__init__::232::IOProcess::(_processLogs) (4758) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:18:13,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3544968-cc0f-458a-8c94-a41621cc6dd9`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:18:13,829::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 14:18:13,829::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00130224 s, 349 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:13,829::__init__::232::IOProcess::(_processLogs) (4758) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:13,833::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:13,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:13,835::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:18:13,837::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:13,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:18:13,841::__init__::232::IOProcess::(_processLogs) (4759) Got request for method 'statvfs' >Thread-86::INFO::2014-08-11 14:18:13,844::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:18:13,845::__init__::232::IOProcess::(_processLogs) (4759) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:13,845::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:18:13,846::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3544968-cc0f-458a-8c94-a41621cc6dd9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:13,847::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3544968-cc0f-458a-8c94-a41621cc6dd9`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:13,849::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:18:13,850::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:18:13,851::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:18:13,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:18:13,852::task::993::Storage.TaskManager.Task::(_decref) Task=`e3544968-cc0f-458a-8c94-a41621cc6dd9`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:18:13,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:13,855::__init__::232::IOProcess::(_processLogs) (4760) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:13,856::__init__::232::IOProcess::(_processLogs) (4760) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:13,856::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:18:13,857::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:13,857::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:13,857::__init__::232::IOProcess::(_processLogs) (4761) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:13,858::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:13,858::__init__::232::IOProcess::(_processLogs) (4762) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:18:13,867::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:18:13,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`64be95f5-27d5-4e24-8b21-42d9e981b772`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:18:13,868::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:18:13,868::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e34d0c6-71d9-4066-86c3-67339d635325`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:18:13,869::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:18:13,869::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:18:13,869::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e34d0c6-71d9-4066-86c3-67339d635325`::Granted request >Thread-34::DEBUG::2014-08-11 14:18:13,870::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`64be95f5-27d5-4e24-8b21-42d9e981b772`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:18:13,870::task::993::Storage.TaskManager.Task::(_decref) Task=`64be95f5-27d5-4e24-8b21-42d9e981b772`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:18:13,890::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085719552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:13,890::task::1191::Storage.TaskManager.Task::(prepare) Task=`64be95f5-27d5-4e24-8b21-42d9e981b772`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085719552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:13,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`64be95f5-27d5-4e24-8b21-42d9e981b772`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:18:13,891::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:18:13,891::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:18:13,891::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:18:13,892::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:18:13,892::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:18:13,892::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:18:13,893::task::993::Storage.TaskManager.Task::(_decref) Task=`64be95f5-27d5-4e24-8b21-42d9e981b772`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:18:14,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:15,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132586 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:17,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:17,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132172 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:18,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:18,858::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:18,859::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:19,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:19,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01317 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:21,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:21,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133014 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:23,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:23,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132033 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:23,860::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:23,861::__init__::232::IOProcess::(_processLogs) (4763) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:18:23,862::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:23,866::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:18:23,905::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:18:23,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed9c49d9-ab81-452a-93dd-eae4e91e42cb`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 14:18:23,928::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000486016 s, 936 kB/s\n'; <rc> = 0 >Thread-86::INFO::2014-08-11 14:18:23,929::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:18:23,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:23,934::__init__::232::IOProcess::(_processLogs) (4764) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:18:23,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:23,936::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:18:23,938::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:23,939::__init__::232::IOProcess::(_processLogs) (4765) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:23,940::__init__::232::IOProcess::(_processLogs) (4765) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:23,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:23,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:23,944::__init__::232::IOProcess::(_processLogs) (4766) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:23,944::__init__::232::IOProcess::(_processLogs) (4766) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:23,947::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::INFO::2014-08-11 14:18:23,948::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:23,950::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:18:23,950::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed9c49d9-ab81-452a-93dd-eae4e91e42cb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:18:23,950::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:18:23,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed9c49d9-ab81-452a-93dd-eae4e91e42cb`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:18:23,952::__init__::232::IOProcess::(_processLogs) (4767) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:18:23,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:18:23,954::__init__::232::IOProcess::(_processLogs) (4767) Queuing response >Thread-86::DEBUG::2014-08-11 14:18:23,954::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:23,955::task::993::Storage.TaskManager.Task::(_decref) Task=`ed9c49d9-ab81-452a-93dd-eae4e91e42cb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:18:23,968::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:18:23,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed9fb0d8-fdc8-409c-8d8a-5f91f90617bb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:18:23,969::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:18:23,970::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5b347513-1bbc-4e69-b83e-ab0a9ab6ad4f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:18:23,970::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:18:23,970::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:18:23,971::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5b347513-1bbc-4e69-b83e-ab0a9ab6ad4f`::Granted request >Thread-34::DEBUG::2014-08-11 14:18:23,971::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ed9fb0d8-fdc8-409c-8d8a-5f91f90617bb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:18:23,971::task::993::Storage.TaskManager.Task::(_decref) Task=`ed9fb0d8-fdc8-409c-8d8a-5f91f90617bb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:18:23,991::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085719552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:23,991::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed9fb0d8-fdc8-409c-8d8a-5f91f90617bb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085719552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:23,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed9fb0d8-fdc8-409c-8d8a-5f91f90617bb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:18:23,992::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:18:23,992::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:18:23,993::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:18:23,993::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:18:23,993::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:18:23,993::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:18:23,994::task::993::Storage.TaskManager.Task::(_decref) Task=`ed9fb0d8-fdc8-409c-8d8a-5f91f90617bb`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:18:24,447::task::595::Storage.TaskManager.Task::(_updateState) Task=`64625213-1fe3-4d8c-a8a7-82b98a7ea7f1`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:18:24,447::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:18:24,447::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000486016', 'lastCheck': '0.5', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:24,448::task::1191::Storage.TaskManager.Task::(prepare) Task=`64625213-1fe3-4d8c-a8a7-82b98a7ea7f1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000486016', 'lastCheck': '0.5', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:24,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`64625213-1fe3-4d8c-a8a7-82b98a7ea7f1`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:24,448::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:18:24,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:24,448::task::993::Storage.TaskManager.Task::(_decref) Task=`64625213-1fe3-4d8c-a8a7-82b98a7ea7f1`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:18:24,467::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:25,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:25,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131546 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:27,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:27,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131785 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:28,964::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:28,965::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:29,010::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:29,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:29,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133532 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:31,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:31,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133391 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:33,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:33,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132462 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:33,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:33,951::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:33,951::__init__::232::IOProcess::(_processLogs) (4768) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:18:33,952::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:18:33,969::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419121 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:33,970::__init__::232::IOProcess::(_processLogs) (4769) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:18:34,005::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:18:34,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:18:34,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`88fce56c-8f39-4c14-ab9d-ed9303f7b4e8`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:18:34,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::INFO::2014-08-11 14:18:34,014::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:18:34,015::__init__::232::IOProcess::(_processLogs) (4770) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:34,016::__init__::232::IOProcess::(_processLogs) (4770) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:34,018::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:34,020::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:34,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:18:34,023::__init__::232::IOProcess::(_processLogs) (4771) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:34,025::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:34,026::__init__::232::IOProcess::(_processLogs) (4772) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:18:34,034::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:34,035::task::1191::Storage.TaskManager.Task::(prepare) Task=`88fce56c-8f39-4c14-ab9d-ed9303f7b4e8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:34,035::task::595::Storage.TaskManager.Task::(_updateState) Task=`88fce56c-8f39-4c14-ab9d-ed9303f7b4e8`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:34,035::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:18:34,035::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:34,036::task::993::Storage.TaskManager.Task::(_decref) Task=`88fce56c-8f39-4c14-ab9d-ed9303f7b4e8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:18:34,046::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:18:34,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`a41a52ca-435e-46d8-9a05-f703f0357673`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:18:34,047::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:18:34,047::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5674c9e2-5740-49e9-a31b-ef7cc9b59e97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:18:34,048::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:18:34,048::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:18:34,048::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5674c9e2-5740-49e9-a31b-ef7cc9b59e97`::Granted request >Thread-34::DEBUG::2014-08-11 14:18:34,049::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a41a52ca-435e-46d8-9a05-f703f0357673`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:18:34,049::task::993::Storage.TaskManager.Task::(_decref) Task=`a41a52ca-435e-46d8-9a05-f703f0357673`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:18:34,069::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:34,069::task::1191::Storage.TaskManager.Task::(prepare) Task=`a41a52ca-435e-46d8-9a05-f703f0357673`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:34,069::task::595::Storage.TaskManager.Task::(_updateState) Task=`a41a52ca-435e-46d8-9a05-f703f0357673`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:18:34,069::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:18:34,070::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:18:34,070::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:18:34,070::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:18:34,071::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:18:34,071::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:18:34,071::task::993::Storage.TaskManager.Task::(_decref) Task=`a41a52ca-435e-46d8-9a05-f703f0357673`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:18:35,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:35,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132519 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:37,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:37,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131885 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:39,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:39,030::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:39,110::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:18:39,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a927719-a1b3-4c79-a12c-3755c02964d9`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:18:39,564::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:18:39,565::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419121', 'lastCheck': '5.5', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:39,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`0a927719-a1b3-4c79-a12c-3755c02964d9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419121', 'lastCheck': '5.5', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:39,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a927719-a1b3-4c79-a12c-3755c02964d9`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:39,565::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:18:39,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:39,566::task::993::Storage.TaskManager.Task::(_decref) Task=`0a927719-a1b3-4c79-a12c-3755c02964d9`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:18:39,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:39,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:39,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132067 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:41,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:41,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131753 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:43,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:43,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131672 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:44,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:44,029::__init__::232::IOProcess::(_processLogs) (4773) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:18:44,029::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:18:44,047::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420535 s, 1.1 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:18:44,083::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:18:44,112::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:18:44,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`600e04d5-33c2-4560-844a-5ce2a028d51e`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:18:44,113::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::INFO::2014-08-11 14:18:44,114::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:18:44,115::__init__::232::IOProcess::(_processLogs) (4774) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:44,120::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:44,122::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:44,124::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:18:44,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:44,125::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:44,126::__init__::232::IOProcess::(_processLogs) (4775) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:44,126::__init__::232::IOProcess::(_processLogs) (4775) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:44,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:44,127::__init__::232::IOProcess::(_processLogs) (4776) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:44,127::__init__::232::IOProcess::(_processLogs) (4776) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:44,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:44,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:44,129::__init__::232::IOProcess::(_processLogs) (4777) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:44,129::__init__::232::IOProcess::(_processLogs) (4777) Queuing response >Thread-86::INFO::2014-08-11 14:18:44,132::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:44,133::task::1191::Storage.TaskManager.Task::(prepare) Task=`600e04d5-33c2-4560-844a-5ce2a028d51e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:44,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`600e04d5-33c2-4560-844a-5ce2a028d51e`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:44,133::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:18:44,134::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:44,134::task::993::Storage.TaskManager.Task::(_decref) Task=`600e04d5-33c2-4560-844a-5ce2a028d51e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:18:44,146::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:18:44,147::task::595::Storage.TaskManager.Task::(_updateState) Task=`67370af5-5852-4122-a06d-fdb7f863ee19`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:18:44,147::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:18:44,148::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1be8cab2-4614-4be3-bdda-c51df22051a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:18:44,148::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:18:44,149::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:18:44,149::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1be8cab2-4614-4be3-bdda-c51df22051a8`::Granted request >Thread-34::DEBUG::2014-08-11 14:18:44,150::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`67370af5-5852-4122-a06d-fdb7f863ee19`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:18:44,150::task::993::Storage.TaskManager.Task::(_decref) Task=`67370af5-5852-4122-a06d-fdb7f863ee19`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:18:44,169::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:44,170::task::1191::Storage.TaskManager.Task::(prepare) Task=`67370af5-5852-4122-a06d-fdb7f863ee19`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:44,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`67370af5-5852-4122-a06d-fdb7f863ee19`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:18:44,170::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:18:44,170::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:18:44,171::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:18:44,171::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:18:44,171::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:18:44,172::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:18:44,172::task::993::Storage.TaskManager.Task::(_decref) Task=`67370af5-5852-4122-a06d-fdb7f863ee19`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:18:45,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:45,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132445 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:47,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:47,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133727 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:49,131::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:49,132::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:49,210::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:49,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:49,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132348 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:51,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:51,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132731 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:53,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:54,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013089 s, 78.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:54,134::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:18:54,184::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:18:54,210::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:54,212::__init__::232::IOProcess::(_processLogs) (4778) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:18:54,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`209ee18a-3d61-42d1-9ea3-eff2aaf7d230`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 14:18:54,215::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::INFO::2014-08-11 14:18:54,216::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:54,235::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:54,237::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:54,238::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:18:54,240::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000899286 s, 506 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:18:54,241::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:54,241::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:54,242::__init__::232::IOProcess::(_processLogs) (4779) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:18:54,244::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:18:54,244::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:54,244::__init__::232::IOProcess::(_processLogs) (4780) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:54,244::__init__::232::IOProcess::(_processLogs) (4780) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:54,246::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:54,246::__init__::232::IOProcess::(_processLogs) (4781) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:54,247::__init__::232::IOProcess::(_processLogs) (4781) Queuing response >Thread-14::DEBUG::2014-08-11 14:18:54,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:18:54,249::__init__::232::IOProcess::(_processLogs) (4782) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:18:54,249::__init__::232::IOProcess::(_processLogs) (4782) Queuing response >Thread-86::INFO::2014-08-11 14:18:54,253::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:54,254::task::1191::Storage.TaskManager.Task::(prepare) Task=`209ee18a-3d61-42d1-9ea3-eff2aaf7d230`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:18:54,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`209ee18a-3d61-42d1-9ea3-eff2aaf7d230`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:54,254::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:18:54,255::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:54,255::task::993::Storage.TaskManager.Task::(_decref) Task=`209ee18a-3d61-42d1-9ea3-eff2aaf7d230`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:18:54,265::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:18:54,266::task::595::Storage.TaskManager.Task::(_updateState) Task=`57c5beb7-1241-419c-b9bb-c8445b9505a7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:18:54,266::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:18:54,267::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1fb426ef-d226-4aba-a5f0-6e4e51b5eaf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:18:54,267::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:18:54,268::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:18:54,268::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1fb426ef-d226-4aba-a5f0-6e4e51b5eaf1`::Granted request >Thread-34::DEBUG::2014-08-11 14:18:54,268::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`57c5beb7-1241-419c-b9bb-c8445b9505a7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:18:54,269::task::993::Storage.TaskManager.Task::(_decref) Task=`57c5beb7-1241-419c-b9bb-c8445b9505a7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:18:54,288::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:54,289::task::1191::Storage.TaskManager.Task::(prepare) Task=`57c5beb7-1241-419c-b9bb-c8445b9505a7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:18:54,289::task::595::Storage.TaskManager.Task::(_updateState) Task=`57c5beb7-1241-419c-b9bb-c8445b9505a7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:18:54,289::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:18:54,289::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:18:54,290::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:18:54,290::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:18:54,290::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:18:54,290::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:18:54,291::task::993::Storage.TaskManager.Task::(_decref) Task=`57c5beb7-1241-419c-b9bb-c8445b9505a7`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:18:54,673::task::595::Storage.TaskManager.Task::(_updateState) Task=`81d6c356-e8af-47c8-8b24-f0207cddc28f`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:18:54,673::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:18:54,674::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000899286', 'lastCheck': '0.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:54,674::task::1191::Storage.TaskManager.Task::(prepare) Task=`81d6c356-e8af-47c8-8b24-f0207cddc28f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000899286', 'lastCheck': '0.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:18:54,674::task::595::Storage.TaskManager.Task::(_updateState) Task=`81d6c356-e8af-47c8-8b24-f0207cddc28f`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:18:54,674::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:18:54,674::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:18:54,675::task::993::Storage.TaskManager.Task::(_decref) Task=`81d6c356-e8af-47c8-8b24-f0207cddc28f`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:18:54,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:18:56,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:56,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132557 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:18:58,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:18:58,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131691 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:59,242::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:59,243::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:18:59,310::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:00,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:00,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131303 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:02,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:02,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0236715 s, 43.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:04,252::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:04,253::__init__::232::IOProcess::(_processLogs) (4783) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:19:04,303::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:19:04,312::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7a3585d-d42f-4a05-9674-262fabebf009`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 14:19:04,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::INFO::2014-08-11 14:19:04,313::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:04,333::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:19:04,334::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00109159 s, 417 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:04,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:04,337::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:04,338::__init__::232::IOProcess::(_processLogs) (4784) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:19:04,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:04,343::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:04,344::__init__::232::IOProcess::(_processLogs) (4785) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:04,344::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:19:04,347::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:04,349::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:19:04,349::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7a3585d-d42f-4a05-9674-262fabebf009`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:04,350::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:19:04,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7a3585d-d42f-4a05-9674-262fabebf009`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:19:04,352::__init__::232::IOProcess::(_processLogs) (4786) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:19:04,353::__init__::232::IOProcess::(_processLogs) (4786) Queuing response >Thread-86::DEBUG::2014-08-11 14:19:04,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:04,352::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:19:04,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:19:04,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:19:04,357::task::993::Storage.TaskManager.Task::(_decref) Task=`c7a3585d-d42f-4a05-9674-262fabebf009`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:19:04,358::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:04,361::__init__::232::IOProcess::(_processLogs) (4787) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:19:04,373::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:19:04,373::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1e3fe02-17ca-42bc-8f2a-00f3252700b1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:19:04,374::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:19:04,374::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4fdd94b-3cc1-4a43-bdc1-609527db02c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:19:04,374::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:19:04,375::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:19:04,375::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4fdd94b-3cc1-4a43-bdc1-609527db02c7`::Granted request >Thread-34::DEBUG::2014-08-11 14:19:04,376::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b1e3fe02-17ca-42bc-8f2a-00f3252700b1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:19:04,376::task::993::Storage.TaskManager.Task::(_decref) Task=`b1e3fe02-17ca-42bc-8f2a-00f3252700b1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:19:04,396::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:04,396::task::1191::Storage.TaskManager.Task::(prepare) Task=`b1e3fe02-17ca-42bc-8f2a-00f3252700b1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:04,396::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1e3fe02-17ca-42bc-8f2a-00f3252700b1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:19:04,396::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:19:04,397::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:19:04,397::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:19:04,397::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:19:04,397::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:19:04,398::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:19:04,398::task::993::Storage.TaskManager.Task::(_decref) Task=`b1e3fe02-17ca-42bc-8f2a-00f3252700b1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:19:04,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:04,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133333 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:06,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:06,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133584 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:08,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:08,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134748 s, 76.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:19:09,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`1319d763-ff5b-46ba-b385-6484e231246c`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:19:09,253::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:19:09,254::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:09,255::__init__::232::IOProcess::(_processLogs) (4788) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:19:09,256::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:09,256::__init__::232::IOProcess::(_processLogs) (4789) Queuing response >Thread-55::INFO::2014-08-11 14:19:09,257::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:19:09,257::task::1191::Storage.TaskManager.Task::(prepare) Task=`1319d763-ff5b-46ba-b385-6484e231246c`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:19:09,257::task::595::Storage.TaskManager.Task::(_updateState) Task=`1319d763-ff5b-46ba-b385-6484e231246c`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:19:09,258::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:19:09,258::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:19:09,258::task::993::Storage.TaskManager.Task::(_decref) Task=`1319d763-ff5b-46ba-b385-6484e231246c`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:09,363::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:09,364::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:09,365::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:19:09,798::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1da3150-46bd-4fa0-ad6c-ad115126279b`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:19:09,798::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:19:09,799::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00109159', 'lastCheck': '5.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:09,799::task::1191::Storage.TaskManager.Task::(prepare) Task=`f1da3150-46bd-4fa0-ad6c-ad115126279b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00109159', 'lastCheck': '5.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:09,799::task::595::Storage.TaskManager.Task::(_updateState) Task=`f1da3150-46bd-4fa0-ad6c-ad115126279b`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:19:09,799::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:19:09,799::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:19:09,800::task::993::Storage.TaskManager.Task::(_decref) Task=`f1da3150-46bd-4fa0-ad6c-ad115126279b`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:19:09,820::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:10,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:10,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132322 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:12,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:12,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132251 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:14,364::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:14,364::__init__::232::IOProcess::(_processLogs) (4790) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:14,372::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:19:14,412::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:19:14,432::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:14,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:19:14,435::task::595::Storage.TaskManager.Task::(_updateState) Task=`a75a638e-1f45-441e-9959-801e6df78b20`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 14:19:14,435::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000795518 s, 572 kB/s\n'; <rc> = 0 >Thread-86::INFO::2014-08-11 14:19:14,437::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:19:14,441::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:14,442::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:14,441::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:14,443::__init__::232::IOProcess::(_processLogs) (4791) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:19:14,447::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:14,447::__init__::232::IOProcess::(_processLogs) (4792) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:19:14,449::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:14,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:14,450::__init__::232::IOProcess::(_processLogs) (4793) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:19:14,456::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:19:14,457::task::1191::Storage.TaskManager.Task::(prepare) Task=`a75a638e-1f45-441e-9959-801e6df78b20`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:14,458::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:19:14,459::task::595::Storage.TaskManager.Task::(_updateState) Task=`a75a638e-1f45-441e-9959-801e6df78b20`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:19:14,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:19:14,460::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:19:14,461::__init__::232::IOProcess::(_processLogs) (4794) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:19:14,461::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:19:14,462::__init__::232::IOProcess::(_processLogs) (4794) Queuing response >Thread-86::DEBUG::2014-08-11 14:19:14,463::task::993::Storage.TaskManager.Task::(_decref) Task=`a75a638e-1f45-441e-9959-801e6df78b20`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:19:14,475::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:19:14,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`a07b5e17-f6d2-41c6-95e9-2a2e38e3d0be`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:19:14,476::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:19:14,477::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1673766b-56eb-437e-9d86-d51bcb96900a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:19:14,477::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:19:14,477::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:19:14,477::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1673766b-56eb-437e-9d86-d51bcb96900a`::Granted request >Thread-34::DEBUG::2014-08-11 14:19:14,478::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a07b5e17-f6d2-41c6-95e9-2a2e38e3d0be`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:19:14,478::task::993::Storage.TaskManager.Task::(_decref) Task=`a07b5e17-f6d2-41c6-95e9-2a2e38e3d0be`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:19:14,498::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:14,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`a07b5e17-f6d2-41c6-95e9-2a2e38e3d0be`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:14,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`a07b5e17-f6d2-41c6-95e9-2a2e38e3d0be`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:19:14,499::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:19:14,499::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:19:14,499::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:19:14,500::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:19:14,500::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:19:14,500::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:19:14,500::task::993::Storage.TaskManager.Task::(_decref) Task=`a07b5e17-f6d2-41c6-95e9-2a2e38e3d0be`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:19:14,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:14,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132208 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:16,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:16,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132806 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:18,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:18,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132947 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:19,449::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:19,451::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:19,510::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:20,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:20,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132581 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:22,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:22,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131802 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:24,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:24,460::__init__::232::IOProcess::(_processLogs) (4795) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:19:24,510::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:24,529::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:19:24,530::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:19:24,534::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c329342-0c5d-4264-a949-e9dd6707c439`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:24,534::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:19:24,536::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 14:19:24,536::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00203171 s, 224 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:24,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:24,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:24,542::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:24,543::__init__::232::IOProcess::(_processLogs) (4796) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:19:24,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:24,547::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:24,548::__init__::232::IOProcess::(_processLogs) (4797) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:19:24,548::__init__::232::IOProcess::(_processLogs) (4797) Queuing response >Thread-14::DEBUG::2014-08-11 14:19:24,551::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:24,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:24,552::__init__::232::IOProcess::(_processLogs) (4798) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:19:24,553::__init__::232::IOProcess::(_processLogs) (4798) Queuing response >Thread-86::INFO::2014-08-11 14:19:24,553::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:19:24,554::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c329342-0c5d-4264-a949-e9dd6707c439`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:24,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:19:24,555::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c329342-0c5d-4264-a949-e9dd6707c439`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:19:24,557::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:19:24,557::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:19:24,558::__init__::232::IOProcess::(_processLogs) (4799) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:19:24,559::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:19:24,560::__init__::232::IOProcess::(_processLogs) (4799) Queuing response >Thread-86::DEBUG::2014-08-11 14:19:24,560::task::993::Storage.TaskManager.Task::(_decref) Task=`4c329342-0c5d-4264-a949-e9dd6707c439`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:19:24,575::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:19:24,576::task::595::Storage.TaskManager.Task::(_updateState) Task=`314357e8-db27-489d-8b7c-e39882ecbd57`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:19:24,576::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:19:24,577::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`14c8cb6a-41a2-4f52-890e-4a690e6232f2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:19:24,577::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:19:24,577::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:19:24,578::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`14c8cb6a-41a2-4f52-890e-4a690e6232f2`::Granted request >Thread-34::DEBUG::2014-08-11 14:19:24,578::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`314357e8-db27-489d-8b7c-e39882ecbd57`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:19:24,578::task::993::Storage.TaskManager.Task::(_decref) Task=`314357e8-db27-489d-8b7c-e39882ecbd57`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:19:24,600::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:24,600::task::1191::Storage.TaskManager.Task::(prepare) Task=`314357e8-db27-489d-8b7c-e39882ecbd57`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085654016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:24,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`314357e8-db27-489d-8b7c-e39882ecbd57`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:19:24,602::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:19:24,602::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:19:24,602::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:19:24,603::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:19:24,603::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:19:24,604::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:19:24,604::task::993::Storage.TaskManager.Task::(_decref) Task=`314357e8-db27-489d-8b7c-e39882ecbd57`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:19:24,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1b19524-d926-41d6-b643-563bb9c1ae86`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:19:24,912::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:19:24,912::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00203171', 'lastCheck': '0.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:24,912::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1b19524-d926-41d6-b643-563bb9c1ae86`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00203171', 'lastCheck': '0.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:24,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1b19524-d926-41d6-b643-563bb9c1ae86`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:19:24,913::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:19:24,913::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:19:24,913::task::993::Storage.TaskManager.Task::(_decref) Task=`e1b19524-d926-41d6-b643-563bb9c1ae86`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:19:24,931::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:25,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:25,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134064 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:27,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:27,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131705 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:29,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:29,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133386 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:29,558::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:29,559::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:29,561::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:31,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:31,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133774 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:33,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:33,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133361 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:34,558::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:34,558::__init__::232::IOProcess::(_processLogs) (4800) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:19:34,559::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:34,567::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:19:34,632::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000663096 s, 686 kB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:19:34,632::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:34,634::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:34,636::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:34,638::__init__::232::IOProcess::(_processLogs) (4801) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:19:34,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ef17c2d-d3ef-4e32-93ec-6f77e377ba0b`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:34,643::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:34,643::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:34,645::__init__::232::IOProcess::(_processLogs) (4802) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:19:34,645::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:19:34,647::__init__::232::IOProcess::(_processLogs) (4802) Queuing response >Thread-14::DEBUG::2014-08-11 14:19:34,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:34,649::__init__::232::IOProcess::(_processLogs) (4803) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:19:34,650::__init__::232::IOProcess::(_processLogs) (4803) Queuing response >Thread-14::DEBUG::2014-08-11 14:19:34,650::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:34,651::__init__::232::IOProcess::(_processLogs) (4804) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:19:34,663::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:19:34,663::task::1191::Storage.TaskManager.Task::(prepare) Task=`6ef17c2d-d3ef-4e32-93ec-6f77e377ba0b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:19:34,663::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ef17c2d-d3ef-4e32-93ec-6f77e377ba0b`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:19:34,664::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:19:34,664::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:19:34,664::task::993::Storage.TaskManager.Task::(_decref) Task=`6ef17c2d-d3ef-4e32-93ec-6f77e377ba0b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:19:34,675::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:19:34,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a1dad9e-5b4d-4a0f-ab3e-da00acb1b5b5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:19:34,675::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:19:34,676::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`12139985-35c8-4e1d-b306-460424131a08`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:19:34,676::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:19:34,677::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:19:34,677::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`12139985-35c8-4e1d-b306-460424131a08`::Granted request >Thread-34::DEBUG::2014-08-11 14:19:34,678::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7a1dad9e-5b4d-4a0f-ab3e-da00acb1b5b5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:19:34,678::task::993::Storage.TaskManager.Task::(_decref) Task=`7a1dad9e-5b4d-4a0f-ab3e-da00acb1b5b5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:19:34,697::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:34,698::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a1dad9e-5b4d-4a0f-ab3e-da00acb1b5b5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:34,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a1dad9e-5b4d-4a0f-ab3e-da00acb1b5b5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:19:34,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:19:34,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:19:34,699::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:19:34,699::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:19:34,699::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:19:34,700::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:19:34,700::task::993::Storage.TaskManager.Task::(_decref) Task=`7a1dad9e-5b4d-4a0f-ab3e-da00acb1b5b5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:19:35,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:35,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132219 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:37,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:37,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132934 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:39,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:39,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131764 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:39,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:39,651::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:39,711::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:19:40,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`4070c8a7-baa4-4a4a-871a-8782b6adb682`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:19:40,016::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:19:40,017::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000663096', 'lastCheck': '5.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:40,017::task::1191::Storage.TaskManager.Task::(prepare) Task=`4070c8a7-baa4-4a4a-871a-8782b6adb682`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000663096', 'lastCheck': '5.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:40,017::task::595::Storage.TaskManager.Task::(_updateState) Task=`4070c8a7-baa4-4a4a-871a-8782b6adb682`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:19:40,017::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:19:40,018::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:19:40,018::task::993::Storage.TaskManager.Task::(_decref) Task=`4070c8a7-baa4-4a4a-871a-8782b6adb682`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:19:40,037::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:41,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:41,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132582 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:43,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:43,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132489 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:44,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:44,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:44,710::__init__::232::IOProcess::(_processLogs) (4805) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:19:44,711::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:19:44,729::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:19:44,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`5eb37ca6-3ff8-439e-a365-adbc02623eef`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:19:44,731::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 14:19:44,734::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00040517 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:44,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:44,735::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:44,737::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:44,738::__init__::232::IOProcess::(_processLogs) (4806) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:44,740::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:44,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::INFO::2014-08-11 14:19:44,746::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:44,747::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:44,747::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:19:44,748::task::1191::Storage.TaskManager.Task::(prepare) Task=`5eb37ca6-3ff8-439e-a365-adbc02623eef`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:44,749::__init__::232::IOProcess::(_processLogs) (4807) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:19:44,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`5eb37ca6-3ff8-439e-a365-adbc02623eef`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:19:44,750::__init__::232::IOProcess::(_processLogs) (4807) Queuing response >Thread-86::DEBUG::2014-08-11 14:19:44,751::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:19:44,753::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:19:44,755::task::993::Storage.TaskManager.Task::(_decref) Task=`5eb37ca6-3ff8-439e-a365-adbc02623eef`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:19:44,755::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:44,757::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:44,758::__init__::232::IOProcess::(_processLogs) (4808) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:19:44,759::__init__::232::IOProcess::(_processLogs) (4808) Queuing response >Thread-14::DEBUG::2014-08-11 14:19:44,759::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:44,759::__init__::232::IOProcess::(_processLogs) (4809) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:19:44,768::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:19:44,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`f583cbf6-a064-477f-87b5-c05abb85f4d8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:19:44,769::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:19:44,770::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b781fd51-c4f6-4763-8371-7827864efd1b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:19:44,770::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:19:44,770::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:19:44,771::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b781fd51-c4f6-4763-8371-7827864efd1b`::Granted request >Thread-34::DEBUG::2014-08-11 14:19:44,771::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f583cbf6-a064-477f-87b5-c05abb85f4d8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:19:44,771::task::993::Storage.TaskManager.Task::(_decref) Task=`f583cbf6-a064-477f-87b5-c05abb85f4d8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:19:44,791::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:44,792::task::1191::Storage.TaskManager.Task::(prepare) Task=`f583cbf6-a064-477f-87b5-c05abb85f4d8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:44,792::task::595::Storage.TaskManager.Task::(_updateState) Task=`f583cbf6-a064-477f-87b5-c05abb85f4d8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:19:44,792::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:19:44,792::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:19:44,793::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:19:44,793::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:19:44,793::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:19:44,793::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:19:44,794::task::993::Storage.TaskManager.Task::(_decref) Task=`f583cbf6-a064-477f-87b5-c05abb85f4d8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:19:45,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:45,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131857 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:47,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:47,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131612 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:49,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:49,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132451 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:49,762::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:49,763::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:49,764::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:51,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:51,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132617 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:53,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:53,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132352 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:19:54,761::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:54,762::__init__::232::IOProcess::(_processLogs) (4810) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:19:54,763::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:54,771::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:19:54,806::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:19:54,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`8198a7de-2276-42a4-aa55-b8e71576db6b`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 14:19:54,830::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000536771 s, 848 kB/s\n'; <rc> = 0 >Thread-86::INFO::2014-08-11 14:19:54,832::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:19:54,834::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:54,836::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:54,838::__init__::232::IOProcess::(_processLogs) (4811) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:54,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:19:54,845::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:54,846::__init__::232::IOProcess::(_processLogs) (4812) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:19:54,846::__init__::232::IOProcess::(_processLogs) (4812) Queuing response >Thread-14::DEBUG::2014-08-11 14:19:54,847::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:19:54,847::__init__::232::IOProcess::(_processLogs) (4813) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:19:54,848::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:54,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:19:54,851::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:19:54,851::task::1191::Storage.TaskManager.Task::(prepare) Task=`8198a7de-2276-42a4-aa55-b8e71576db6b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:19:54,852::__init__::232::IOProcess::(_processLogs) (4814) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:19:54,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`8198a7de-2276-42a4-aa55-b8e71576db6b`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:19:54,853::__init__::232::IOProcess::(_processLogs) (4814) Queuing response >Thread-86::DEBUG::2014-08-11 14:19:54,853::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:19:54,854::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:19:54,854::task::993::Storage.TaskManager.Task::(_decref) Task=`8198a7de-2276-42a4-aa55-b8e71576db6b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:19:54,866::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:19:54,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`f537d2d2-e175-4919-956d-287b03baeeb0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:19:54,867::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:19:54,868::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c61aaa01-4bb1-4ffb-a770-f28ff4770853`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:19:54,868::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:19:54,868::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:19:54,869::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c61aaa01-4bb1-4ffb-a770-f28ff4770853`::Granted request >Thread-34::DEBUG::2014-08-11 14:19:54,869::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f537d2d2-e175-4919-956d-287b03baeeb0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:19:54,869::task::993::Storage.TaskManager.Task::(_decref) Task=`f537d2d2-e175-4919-956d-287b03baeeb0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:19:54,889::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:54,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`f537d2d2-e175-4919-956d-287b03baeeb0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:19:54,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`f537d2d2-e175-4919-956d-287b03baeeb0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:19:54,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:19:54,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:19:54,890::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:19:54,891::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:19:54,891::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:19:54,891::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:19:54,891::task::993::Storage.TaskManager.Task::(_decref) Task=`f537d2d2-e175-4919-956d-287b03baeeb0`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:19:55,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`21b6f92c-f503-487d-ad99-89de4955b0b4`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:19:55,135::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:19:55,135::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000536771', 'lastCheck': '0.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:55,135::task::1191::Storage.TaskManager.Task::(prepare) Task=`21b6f92c-f503-487d-ad99-89de4955b0b4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000536771', 'lastCheck': '0.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:19:55,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`21b6f92c-f503-487d-ad99-89de4955b0b4`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:19:55,136::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:19:55,136::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:19:55,136::task::993::Storage.TaskManager.Task::(_decref) Task=`21b6f92c-f503-487d-ad99-89de4955b0b4`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:19:55,156::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:55,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:55,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01314 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:19:57,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:57,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013197 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:59,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:59,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:19:59,852::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:19:59,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:19:59,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134772 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:02,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:02,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283878 s, 36.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:04,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:04,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133042 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:04,851::__init__::232::IOProcess::(_processLogs) (4815) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:20:04,852::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:04,859::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:20:04,904::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:04,930::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:04,931::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:20:04,933::task::595::Storage.TaskManager.Task::(_updateState) Task=`aab8fdd8-6af6-48e8-b368-35dcc0d35831`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:04,934::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 14:20:04,935::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00468372 s, 97.1 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:04,936::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:04,937::__init__::232::IOProcess::(_processLogs) (4816) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:04,938::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:04,938::__init__::232::IOProcess::(_processLogs) (4817) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:04,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:04,940::__init__::232::IOProcess::(_processLogs) (4818) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:04,941::__init__::232::IOProcess::(_processLogs) (4818) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:04,942::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:04,942::__init__::232::IOProcess::(_processLogs) (4819) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:20:04,951::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:20:04,953::task::1191::Storage.TaskManager.Task::(prepare) Task=`aab8fdd8-6af6-48e8-b368-35dcc0d35831`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:20:04,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`aab8fdd8-6af6-48e8-b368-35dcc0d35831`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:04,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:04,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:04,953::task::993::Storage.TaskManager.Task::(_decref) Task=`aab8fdd8-6af6-48e8-b368-35dcc0d35831`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:20:04,966::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:20:04,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`31985777-40dc-46c9-bc1a-a42380cfeed1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:20:04,967::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:20:04,968::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9514bd8d-0a24-4f96-90a9-8d33d9b1cdd6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:20:04,968::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:20:04,968::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:20:04,968::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9514bd8d-0a24-4f96-90a9-8d33d9b1cdd6`::Granted request >Thread-34::DEBUG::2014-08-11 14:20:04,969::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`31985777-40dc-46c9-bc1a-a42380cfeed1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:20:04,969::task::993::Storage.TaskManager.Task::(_decref) Task=`31985777-40dc-46c9-bc1a-a42380cfeed1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:20:04,989::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:04,989::task::1191::Storage.TaskManager.Task::(prepare) Task=`31985777-40dc-46c9-bc1a-a42380cfeed1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:04,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`31985777-40dc-46c9-bc1a-a42380cfeed1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:20:04,990::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:20:04,990::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:20:04,990::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:20:04,991::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:20:04,991::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:20:04,991::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:20:04,991::task::993::Storage.TaskManager.Task::(_decref) Task=`31985777-40dc-46c9-bc1a-a42380cfeed1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:20:06,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:06,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135286 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:08,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:08,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131347 s, 78.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:20:09,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1f78304-ab98-4e55-a032-0ebf4f7c4888`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:20:09,414::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:20:09,415::__init__::232::IOProcess::(_processLogs) (4820) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:20:09,417::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:09,417::__init__::232::IOProcess::(_processLogs) (4821) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:20:09,418::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:20:09,418::task::1191::Storage.TaskManager.Task::(prepare) Task=`b1f78304-ab98-4e55-a032-0ebf4f7c4888`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:20:09,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1f78304-ab98-4e55-a032-0ebf4f7c4888`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:20:09,418::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:20:09,418::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:20:09,419::task::993::Storage.TaskManager.Task::(_decref) Task=`b1f78304-ab98-4e55-a032-0ebf4f7c4888`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:09,939::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:09,940::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:10,011::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:20:10,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ade0bf0-dba3-49ab-9a37-67513f1d9d88`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:10,245::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:20:10,245::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00468372', 'lastCheck': '5.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:10,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`8ade0bf0-dba3-49ab-9a37-67513f1d9d88`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00468372', 'lastCheck': '5.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:10,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ade0bf0-dba3-49ab-9a37-67513f1d9d88`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:10,246::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:10,246::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:10,247::task::993::Storage.TaskManager.Task::(_decref) Task=`8ade0bf0-dba3-49ab-9a37-67513f1d9d88`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:20:10,265::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:20:10,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:10,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135138 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:12,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:12,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133157 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:14,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:14,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131653 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:14,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:14,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:14,954::__init__::232::IOProcess::(_processLogs) (4822) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:20:15,006::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-14::DEBUG::2014-08-11 14:20:15,012::__init__::232::IOProcess::(_processLogs) DEBUG|(4822) Queuing response >Thread-16::DEBUG::2014-08-11 14:20:15,013::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:20:15,014::task::595::Storage.TaskManager.Task::(_updateState) Task=`23825f58-aa6c-4677-a08a-35cf052c6948`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:15,031::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:15,034::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:20:15,035::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000787489 s, 578 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:15,038::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:15,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:15,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:15,042::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:15,043::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:15,044::__init__::232::IOProcess::(_processLogs) (4823) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:15,047::__init__::232::IOProcess::(_processLogs) DEBUG|(4823) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:15,049::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:15,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:15,050::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:15,051::__init__::232::IOProcess::(_processLogs) (4824) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:15,051::__init__::232::IOProcess::(_processLogs) (4824) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:15,052::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:15,052::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:15,052::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::INFO::2014-08-11 14:20:15,053::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:20:15,053::__init__::232::IOProcess::(_processLogs) (4825) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:20:15,054::task::1191::Storage.TaskManager.Task::(prepare) Task=`23825f58-aa6c-4677-a08a-35cf052c6948`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:20:15,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`23825f58-aa6c-4677-a08a-35cf052c6948`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:20:15,055::__init__::232::IOProcess::(_processLogs) DEBUG|(4825) Queuing response >Thread-86::DEBUG::2014-08-11 14:20:15,056::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:20:15,057::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:20:15,057::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:20:15,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:20:15,059::task::993::Storage.TaskManager.Task::(_decref) Task=`23825f58-aa6c-4677-a08a-35cf052c6948`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:20:15,060::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:15,062::__init__::232::IOProcess::(_processLogs) (4826) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:15,063::__init__::232::IOProcess::(_processLogs) DEBUG|(4826) Queuing response >Thread-34::DEBUG::2014-08-11 14:20:15,071::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-34::DEBUG::2014-08-11 14:20:15,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`06a9d365-fa6f-42ef-a003-99bf6e3e5e9e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:20:15,072::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:20:15,073::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce9481d8-52e7-43b8-ab05-e874ac80914d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:20:15,073::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:20:15,074::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:20:15,074::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce9481d8-52e7-43b8-ab05-e874ac80914d`::Granted request >Thread-34::DEBUG::2014-08-11 14:20:15,074::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`06a9d365-fa6f-42ef-a003-99bf6e3e5e9e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:20:15,075::task::993::Storage.TaskManager.Task::(_decref) Task=`06a9d365-fa6f-42ef-a003-99bf6e3e5e9e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:20:15,094::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:15,095::task::1191::Storage.TaskManager.Task::(prepare) Task=`06a9d365-fa6f-42ef-a003-99bf6e3e5e9e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:15,095::task::595::Storage.TaskManager.Task::(_updateState) Task=`06a9d365-fa6f-42ef-a003-99bf6e3e5e9e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:20:15,095::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:20:15,095::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:20:15,096::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:20:15,096::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:20:15,096::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:20:15,096::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:20:15,097::task::993::Storage.TaskManager.Task::(_decref) Task=`06a9d365-fa6f-42ef-a003-99bf6e3e5e9e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:20:16,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:16,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132086 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:18,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:18,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131929 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:20,050::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:20,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:20,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:20:20,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:20,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133199 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:22,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:22,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131593 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:24,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:24,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131987 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:25,065::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:25,066::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:20:25,067::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-86::DEBUG::2014-08-11 14:20:25,109::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:20:25,111::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:20:25,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc4a5694-6154-4beb-866e-c2dd794efaf8`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:20:25,132::__init__::232::IOProcess::(_processLogs) (4827) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:25,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:20:25,135::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000870851 s, 522 kB/s\n'; <rc> = 0 >Thread-86::INFO::2014-08-11 14:20:25,135::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:20:25,136::__init__::232::IOProcess::(_processLogs) (4827) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:25,140::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:25,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:25,143::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:25,144::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:25,145::__init__::232::IOProcess::(_processLogs) (4828) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:25,147::__init__::232::IOProcess::(_processLogs) DEBUG|(4828) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:25,149::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:25,150::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:25,151::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:25,151::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:25,152::__init__::232::IOProcess::(_processLogs) (4829) Got request for method 'access' >Thread-86::INFO::2014-08-11 14:20:25,154::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:20:25,154::__init__::232::IOProcess::(_processLogs) (4829) Queuing response >Thread-86::DEBUG::2014-08-11 14:20:25,155::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc4a5694-6154-4beb-866e-c2dd794efaf8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:20:25,157::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-86::DEBUG::2014-08-11 14:20:25,157::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc4a5694-6154-4beb-866e-c2dd794efaf8`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:25,159::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:20:25,160::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:20:25,160::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:20:25,161::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-86::DEBUG::2014-08-11 14:20:25,162::task::993::Storage.TaskManager.Task::(_decref) Task=`cc4a5694-6154-4beb-866e-c2dd794efaf8`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:20:25,162::__init__::232::IOProcess::(_processLogs) (4830) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:25,164::__init__::232::IOProcess::(_processLogs) (4830) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:25,165::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:25,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:25,167::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:25,167::__init__::232::IOProcess::(_processLogs) (4831) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:25,168::__init__::232::IOProcess::(_processLogs) (4831) Queuing response >Thread-34::DEBUG::2014-08-11 14:20:25,175::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:20:25,175::task::595::Storage.TaskManager.Task::(_updateState) Task=`4643d9fc-536f-44af-b3b7-f76a441f0e5f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:20:25,176::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:20:25,176::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`76a9e6ca-d3f5-4244-9039-f88d1eb11834`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:20:25,177::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:20:25,177::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:20:25,177::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`76a9e6ca-d3f5-4244-9039-f88d1eb11834`::Granted request >Thread-34::DEBUG::2014-08-11 14:20:25,178::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4643d9fc-536f-44af-b3b7-f76a441f0e5f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:20:25,178::task::993::Storage.TaskManager.Task::(_decref) Task=`4643d9fc-536f-44af-b3b7-f76a441f0e5f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:20:25,198::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:25,198::task::1191::Storage.TaskManager.Task::(prepare) Task=`4643d9fc-536f-44af-b3b7-f76a441f0e5f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:25,198::task::595::Storage.TaskManager.Task::(_updateState) Task=`4643d9fc-536f-44af-b3b7-f76a441f0e5f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:20:25,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:20:25,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:20:25,199::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:20:25,200::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:20:25,200::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:20:25,200::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:20:25,200::task::993::Storage.TaskManager.Task::(_decref) Task=`4643d9fc-536f-44af-b3b7-f76a441f0e5f`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:20:25,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c1379ce-cdaa-4cbd-abbe-e2ab2f915b8c`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:25,351::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:20:25,352::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000870851', 'lastCheck': '0.2', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:25,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c1379ce-cdaa-4cbd-abbe-e2ab2f915b8c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000870851', 'lastCheck': '0.2', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:25,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c1379ce-cdaa-4cbd-abbe-e2ab2f915b8c`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:25,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:25,353::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:25,353::task::993::Storage.TaskManager.Task::(_decref) Task=`8c1379ce-cdaa-4cbd-abbe-e2ab2f915b8c`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:20:25,376::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:20:26,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:26,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131393 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:28,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:28,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132187 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:30,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:30,165::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:30,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:20:30,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:31,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133229 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:33,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:33,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013244 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:35,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:35,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131118 s, 78.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:35,168::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:35,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:20:35,170::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:20:35,213::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:35,232::__init__::232::IOProcess::(_processLogs) (4832) Got request for method 'statvfs' >Thread-86::DEBUG::2014-08-11 14:20:35,233::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:35,235::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:20:35,236::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00089509 s, 508 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:35,236::__init__::232::IOProcess::(_processLogs) (4832) Queuing response >Thread-86::DEBUG::2014-08-11 14:20:35,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9ee9316-7d80-46b4-82d0-c30ed3d0093e`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:35,242::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:20:35,244::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:35,244::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:35,245::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:35,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:35,248::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:35,249::__init__::232::IOProcess::(_processLogs) (4833) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:35,250::__init__::232::IOProcess::(_processLogs) (4833) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:35,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:35,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:35,254::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:35,255::__init__::232::IOProcess::(_processLogs) (4834) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:35,256::__init__::232::IOProcess::(_processLogs) (4834) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:35,257::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:35,258::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:35,258::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:35,258::__init__::232::IOProcess::(_processLogs) (4835) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:35,259::__init__::232::IOProcess::(_processLogs) (4835) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:35,259::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-86::INFO::2014-08-11 14:20:35,260::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:20:35,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:20:35,261::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9ee9316-7d80-46b4-82d0-c30ed3d0093e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 14:20:35,262::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:35,263::__init__::232::IOProcess::(_processLogs) (4836) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:35,263::__init__::232::IOProcess::(_processLogs) (4836) Queuing response >Thread-86::DEBUG::2014-08-11 14:20:35,263::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9ee9316-7d80-46b4-82d0-c30ed3d0093e`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:35,264::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:35,264::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:35,264::task::993::Storage.TaskManager.Task::(_decref) Task=`d9ee9316-7d80-46b4-82d0-c30ed3d0093e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:20:35,275::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:20:35,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`aef036d4-82d2-443c-b850-1753e03f4eae`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:20:35,276::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:20:35,279::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8af4263f-24d2-4eb4-9095-ccfa1689f85f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:20:35,279::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:20:35,280::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:20:35,280::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8af4263f-24d2-4eb4-9095-ccfa1689f85f`::Granted request >Thread-34::DEBUG::2014-08-11 14:20:35,280::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aef036d4-82d2-443c-b850-1753e03f4eae`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:20:35,281::task::993::Storage.TaskManager.Task::(_decref) Task=`aef036d4-82d2-443c-b850-1753e03f4eae`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:20:35,301::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:35,302::task::1191::Storage.TaskManager.Task::(prepare) Task=`aef036d4-82d2-443c-b850-1753e03f4eae`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:35,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`aef036d4-82d2-443c-b850-1753e03f4eae`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:20:35,305::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:20:35,307::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:20:35,309::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:20:35,312::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:20:35,313::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:20:35,315::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:20:35,317::task::993::Storage.TaskManager.Task::(_decref) Task=`aef036d4-82d2-443c-b850-1753e03f4eae`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:20:37,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:37,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013272 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:39,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:39,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132419 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:40,263::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:40,264::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:40,312::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:20:40,472::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1961346-be6d-48a2-a4ab-9484a6abe9aa`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:40,472::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:20:40,473::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00089509', 'lastCheck': '5.2', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:40,473::task::1191::Storage.TaskManager.Task::(prepare) Task=`b1961346-be6d-48a2-a4ab-9484a6abe9aa`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00089509', 'lastCheck': '5.2', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:40,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`b1961346-be6d-48a2-a4ab-9484a6abe9aa`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:40,473::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:40,474::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:40,474::task::993::Storage.TaskManager.Task::(_decref) Task=`b1961346-be6d-48a2-a4ab-9484a6abe9aa`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:20:40,495::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:20:41,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:41,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133045 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:43,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:43,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131714 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:45,263::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:45,264::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:45,264::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:45,264::__init__::232::IOProcess::(_processLogs) (4837) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:45,312::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4837) Queuing response >Thread-16::DEBUG::2014-08-11 14:20:45,313::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:45,331::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:45,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:45,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:20:45,335::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417376 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:45,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:45,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-86::DEBUG::2014-08-11 14:20:45,339::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:20:45,340::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:45,341::__init__::232::IOProcess::(_processLogs) (4838) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:45,343::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4838) Queuing response >Thread-86::DEBUG::2014-08-11 14:20:45,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`47eef10f-a82b-440e-a894-2ccd98aadafb`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:20:45,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-86::INFO::2014-08-11 14:20:45,346::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:20:45,347::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:45,348::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:45,348::__init__::232::IOProcess::(_processLogs) (4839) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:45,348::__init__::232::IOProcess::(_processLogs) (4839) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:45,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:45,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:45,350::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:45,350::__init__::232::IOProcess::(_processLogs) (4840) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:45,351::__init__::232::IOProcess::(_processLogs) (4840) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:45,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:45,353::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:45,354::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:45,355::__init__::232::IOProcess::(_processLogs) (4841) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:45,355::__init__::232::IOProcess::(_processLogs) (4841) Queuing response >Thread-86::INFO::2014-08-11 14:20:45,363::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:20:45,364::task::1191::Storage.TaskManager.Task::(prepare) Task=`47eef10f-a82b-440e-a894-2ccd98aadafb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:20:45,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`47eef10f-a82b-440e-a894-2ccd98aadafb`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:45,364::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:45,364::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:45,364::task::993::Storage.TaskManager.Task::(_decref) Task=`47eef10f-a82b-440e-a894-2ccd98aadafb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:20:45,374::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:20:45,375::task::595::Storage.TaskManager.Task::(_updateState) Task=`8dd03473-0c2d-4c28-bb7f-7ae77658a73b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:20:45,375::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:20:45,376::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c448956c-3264-4530-82d2-4aca4889dad4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:20:45,376::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:20:45,377::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:20:45,377::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c448956c-3264-4530-82d2-4aca4889dad4`::Granted request >Thread-34::DEBUG::2014-08-11 14:20:45,377::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8dd03473-0c2d-4c28-bb7f-7ae77658a73b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:20:45,378::task::993::Storage.TaskManager.Task::(_decref) Task=`8dd03473-0c2d-4c28-bb7f-7ae77658a73b`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:20:45,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:45,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0353181 s, 29.0 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 14:20:45,471::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:45,474::task::1191::Storage.TaskManager.Task::(prepare) Task=`8dd03473-0c2d-4c28-bb7f-7ae77658a73b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:45,477::task::595::Storage.TaskManager.Task::(_updateState) Task=`8dd03473-0c2d-4c28-bb7f-7ae77658a73b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:20:45,479::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:20:45,482::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:20:45,487::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:20:45,491::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:20:45,492::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:20:45,493::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:20:45,495::task::993::Storage.TaskManager.Task::(_decref) Task=`8dd03473-0c2d-4c28-bb7f-7ae77658a73b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:20:47,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:47,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131919 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:49,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:49,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136577 s, 75.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:50,336::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:50,337::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:50,413::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:20:51,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:51,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132801 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:53,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:53,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131742 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:20:55,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:55,355::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:20:55,356::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:20:55,412::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:55,429::__init__::232::IOProcess::(_processLogs) (4842) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:55,431::__init__::232::IOProcess::(_processLogs) (4842) Queuing response >Thread-16::DEBUG::2014-08-11 14:20:55,432::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0012721 s, 358 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:55,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:55,436::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:55,438::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:55,439::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:55,439::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:55,441::__init__::232::IOProcess::(_processLogs) (4843) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:20:55,443::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4843) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:20:55,445::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:20:55,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:55,449::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:55,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:55,450::__init__::232::IOProcess::(_processLogs) (4844) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:55,450::__init__::232::IOProcess::(_processLogs) (4844) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:55,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:55,451::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:55,452::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:55,452::__init__::232::IOProcess::(_processLogs) (4845) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:55,452::__init__::232::IOProcess::(_processLogs) (4845) Queuing response >Thread-14::DEBUG::2014-08-11 14:20:55,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:20:55,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:20:55,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:20:55,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(4846) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:20:55,454::__init__::232::IOProcess::(_processLogs) (4846) Queuing response >Thread-86::DEBUG::2014-08-11 14:20:55,517::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:20:55,518::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e9bdbba-ae48-442d-beb6-9d0c7885e24d`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:55,518::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:20:55,531::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:20:55,531::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e9bdbba-ae48-442d-beb6-9d0c7885e24d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:20:55,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e9bdbba-ae48-442d-beb6-9d0c7885e24d`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:55,532::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:55,532::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:55,532::task::993::Storage.TaskManager.Task::(_decref) Task=`5e9bdbba-ae48-442d-beb6-9d0c7885e24d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:20:55,543::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:20:55,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`770c3ab0-0e74-4e42-897e-bfd10541e4ab`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:20:55,546::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:20:55,549::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d05b6fa9-6ea9-44ba-92c7-34d31252f5a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:20:55,549::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:20:55,550::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:20:55,551::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d05b6fa9-6ea9-44ba-92c7-34d31252f5a9`::Granted request >Thread-34::DEBUG::2014-08-11 14:20:55,554::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`770c3ab0-0e74-4e42-897e-bfd10541e4ab`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:20:55,554::task::993::Storage.TaskManager.Task::(_decref) Task=`770c3ab0-0e74-4e42-897e-bfd10541e4ab`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:20:55,576::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:55,576::task::1191::Storage.TaskManager.Task::(prepare) Task=`770c3ab0-0e74-4e42-897e-bfd10541e4ab`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085588480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:20:55,577::task::595::Storage.TaskManager.Task::(_updateState) Task=`770c3ab0-0e74-4e42-897e-bfd10541e4ab`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:20:55,577::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:20:55,577::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:20:55,577::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:20:55,578::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:20:55,578::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:20:55,578::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:20:55,578::task::993::Storage.TaskManager.Task::(_decref) Task=`770c3ab0-0e74-4e42-897e-bfd10541e4ab`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:20:55,590::task::595::Storage.TaskManager.Task::(_updateState) Task=`46cba35f-64ae-4e2c-879a-5668b23cb2ae`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:20:55,591::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:20:55,591::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0012721', 'lastCheck': '0.1', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:55,591::task::1191::Storage.TaskManager.Task::(prepare) Task=`46cba35f-64ae-4e2c-879a-5668b23cb2ae`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0012721', 'lastCheck': '0.1', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:20:55,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`46cba35f-64ae-4e2c-879a-5668b23cb2ae`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:20:55,592::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:20:55,592::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:20:55,592::task::993::Storage.TaskManager.Task::(_decref) Task=`46cba35f-64ae-4e2c-879a-5668b23cb2ae`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:20:55,610::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:20:55,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:55,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132716 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:20:57,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:20:57,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132886 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:00,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:00,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132757 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:00,453::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:00,454::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:00,512::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:21:02,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:02,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0252141 s, 40.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:04,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:04,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01334 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:05,455::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:05,456::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:05,456::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:05,457::__init__::232::IOProcess::(_processLogs) (4847) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:05,457::__init__::232::IOProcess::(_processLogs) (4847) Queuing response >Thread-16::DEBUG::2014-08-11 14:21:05,457::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:21:05,475::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425953 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:05,476::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:05,477::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:05,477::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:05,477::__init__::232::IOProcess::(_processLogs) (4848) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:05,478::__init__::232::IOProcess::(_processLogs) DEBUG|(4848) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:05,479::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:05,480::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:05,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:05,480::__init__::232::IOProcess::(_processLogs) (4849) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:05,481::__init__::232::IOProcess::(_processLogs) DEBUG|(4849) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:05,482::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:05,482::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:05,483::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:05,483::__init__::232::IOProcess::(_processLogs) (4850) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:05,483::__init__::232::IOProcess::(_processLogs) (4850) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:05,484::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:05,484::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:05,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:05,484::__init__::232::IOProcess::(_processLogs) (4851) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:05,485::__init__::232::IOProcess::(_processLogs) DEBUG|(4851) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:05,519::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:05,520::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:05,521::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:21:05,597::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:21:05,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`68f2dd73-204c-4f63-9058-43170d5ab725`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:05,601::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:21:05,615::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:05,616::task::1191::Storage.TaskManager.Task::(prepare) Task=`68f2dd73-204c-4f63-9058-43170d5ab725`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:05,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`68f2dd73-204c-4f63-9058-43170d5ab725`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:05,616::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:05,616::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:05,617::task::993::Storage.TaskManager.Task::(_decref) Task=`68f2dd73-204c-4f63-9058-43170d5ab725`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:21:05,627::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:21:05,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`34143e8b-6050-45da-a06e-817cb205c202`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:21:05,628::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:21:05,629::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f1a95fbb-e7ca-4110-9feb-be892047b163`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:21:05,629::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:21:05,629::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:21:05,629::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f1a95fbb-e7ca-4110-9feb-be892047b163`::Granted request >Thread-34::DEBUG::2014-08-11 14:21:05,630::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`34143e8b-6050-45da-a06e-817cb205c202`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:21:05,630::task::993::Storage.TaskManager.Task::(_decref) Task=`34143e8b-6050-45da-a06e-817cb205c202`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:21:05,650::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:05,651::task::1191::Storage.TaskManager.Task::(prepare) Task=`34143e8b-6050-45da-a06e-817cb205c202`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:05,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`34143e8b-6050-45da-a06e-817cb205c202`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:21:05,651::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:21:05,651::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:21:05,652::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:21:05,652::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:21:05,652::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:21:05,652::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:21:05,653::task::993::Storage.TaskManager.Task::(_decref) Task=`34143e8b-6050-45da-a06e-817cb205c202`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:21:06,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:06,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131797 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:08,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:08,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133061 s, 77.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:21:09,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`855fc4af-ebca-45bf-80c2-2f3238e32481`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:21:09,525::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:21:09,526::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:09,526::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:09,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:09,527::__init__::232::IOProcess::(_processLogs) DEBUG|(4852) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:21:09,527::__init__::232::IOProcess::(_processLogs) DEBUG|(4852) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:09,528::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:09,528::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:09,529::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:09,529::__init__::232::IOProcess::(_processLogs) (4853) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:21:09,529::__init__::232::IOProcess::(_processLogs) DEBUG|(4853) Queuing response >Thread-55::INFO::2014-08-11 14:21:09,530::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:21:09,530::task::1191::Storage.TaskManager.Task::(prepare) Task=`855fc4af-ebca-45bf-80c2-2f3238e32481`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:21:09,530::task::595::Storage.TaskManager.Task::(_updateState) Task=`855fc4af-ebca-45bf-80c2-2f3238e32481`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:21:09,530::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:21:09,531::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:21:09,531::task::993::Storage.TaskManager.Task::(_decref) Task=`855fc4af-ebca-45bf-80c2-2f3238e32481`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:21:10,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:10,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133017 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:10,530::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:10,534::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:10,539::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:21:10,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`72f6de6a-702e-4390-b43a-ff6c28d8b423`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:10,700::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:21:10,700::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425953', 'lastCheck': '5.2', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:21:10,700::task::1191::Storage.TaskManager.Task::(prepare) Task=`72f6de6a-702e-4390-b43a-ff6c28d8b423`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425953', 'lastCheck': '5.2', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:21:10,701::task::595::Storage.TaskManager.Task::(_updateState) Task=`72f6de6a-702e-4390-b43a-ff6c28d8b423`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:10,701::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:10,701::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:10,701::task::993::Storage.TaskManager.Task::(_decref) Task=`72f6de6a-702e-4390-b43a-ff6c28d8b423`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:21:10,720::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:21:12,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:12,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132542 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:14,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:14,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132082 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:15,487::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:15,488::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:15,488::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:15,488::__init__::232::IOProcess::(_processLogs) (4854) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:15,489::__init__::232::IOProcess::(_processLogs) (4854) Queuing response >Thread-16::DEBUG::2014-08-11 14:21:15,490::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:21:15,507::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423177 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:15,509::__init__::232::IOProcess::(_processLogs) (4855) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:15,513::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:15,514::__init__::232::IOProcess::(_processLogs) (4856) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:15,514::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(4856) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:15,515::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:15,515::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:15,515::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:15,516::__init__::232::IOProcess::(_processLogs) (4857) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:15,516::__init__::232::IOProcess::(_processLogs) (4857) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:15,517::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:15,517::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:15,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:15,519::__init__::232::IOProcess::(_processLogs) (4858) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:15,519::__init__::232::IOProcess::(_processLogs) (4858) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:15,548::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:15,549::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:15,550::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:21:15,666::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:21:15,666::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b674798-14e3-417d-85e3-9353d935835d`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:15,667::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:21:15,679::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:15,680::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b674798-14e3-417d-85e3-9353d935835d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:15,680::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b674798-14e3-417d-85e3-9353d935835d`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:15,680::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:15,680::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:15,681::task::993::Storage.TaskManager.Task::(_decref) Task=`6b674798-14e3-417d-85e3-9353d935835d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:21:15,691::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:21:15,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe633f7e-b7c1-495a-b4cc-f0c34be9c289`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:21:15,692::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:21:15,693::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b432f551-d77a-4056-931f-73d8c95e2153`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:21:15,693::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:21:15,694::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:21:15,694::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b432f551-d77a-4056-931f-73d8c95e2153`::Granted request >Thread-34::DEBUG::2014-08-11 14:21:15,695::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fe633f7e-b7c1-495a-b4cc-f0c34be9c289`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:21:15,695::task::993::Storage.TaskManager.Task::(_decref) Task=`fe633f7e-b7c1-495a-b4cc-f0c34be9c289`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:21:15,714::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:15,715::task::1191::Storage.TaskManager.Task::(prepare) Task=`fe633f7e-b7c1-495a-b4cc-f0c34be9c289`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:15,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe633f7e-b7c1-495a-b4cc-f0c34be9c289`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:21:15,715::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:21:15,715::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:21:15,716::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:21:15,716::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:21:15,716::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:21:15,717::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:21:15,717::task::993::Storage.TaskManager.Task::(_decref) Task=`fe633f7e-b7c1-495a-b4cc-f0c34be9c289`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:21:16,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:16,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148412 s, 69.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:18,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:18,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130858 s, 78.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:20,557::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:20,558::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:20,614::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:21:20,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:20,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132622 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:22,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:22,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131414 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:24,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:24,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131388 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:25,519::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:25,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:25,521::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:21:25,522::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:21:25,523::__init__::232::IOProcess::(_processLogs) (4859) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:25,539::__init__::232::IOProcess::(_processLogs) (4859) Queuing response >Thread-16::DEBUG::2014-08-11 14:21:25,541::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000582048 s, 782 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:25,542::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:25,542::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:25,542::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:25,543::__init__::232::IOProcess::(_processLogs) (4860) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:25,544::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:25,544::__init__::232::IOProcess::(_processLogs) (4861) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:25,545::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:25,545::__init__::232::IOProcess::(_processLogs) (4862) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:25,546::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:25,547::__init__::232::IOProcess::(_processLogs) (4863) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:25,547::__init__::232::IOProcess::(_processLogs) (4863) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:25,618::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:25,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:25,714::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:21:25,730::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:21:25,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`40cadda1-579b-4ec4-9ef2-bd59866f627b`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:25,731::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:21:25,744::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:25,744::task::1191::Storage.TaskManager.Task::(prepare) Task=`40cadda1-579b-4ec4-9ef2-bd59866f627b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:25,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`40cadda1-579b-4ec4-9ef2-bd59866f627b`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:25,745::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:25,745::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:25,745::task::993::Storage.TaskManager.Task::(_decref) Task=`40cadda1-579b-4ec4-9ef2-bd59866f627b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:21:25,755::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:21:25,756::task::595::Storage.TaskManager.Task::(_updateState) Task=`09e61eb0-b712-4171-aa3e-939214a2ae30`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:21:25,756::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:21:25,757::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3304124f-4912-4c75-9520-6f98811123ac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:21:25,757::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:21:25,758::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:21:25,758::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3304124f-4912-4c75-9520-6f98811123ac`::Granted request >Thread-34::DEBUG::2014-08-11 14:21:25,758::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`09e61eb0-b712-4171-aa3e-939214a2ae30`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:21:25,759::task::993::Storage.TaskManager.Task::(_decref) Task=`09e61eb0-b712-4171-aa3e-939214a2ae30`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:21:25,778::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:25,779::task::1191::Storage.TaskManager.Task::(prepare) Task=`09e61eb0-b712-4171-aa3e-939214a2ae30`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:25,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`09e61eb0-b712-4171-aa3e-939214a2ae30`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:21:25,779::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:21:25,779::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:21:25,780::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:21:25,780::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:21:25,780::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:21:25,780::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:21:25,781::task::993::Storage.TaskManager.Task::(_decref) Task=`09e61eb0-b712-4171-aa3e-939214a2ae30`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:21:25,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`88ac2c13-86ca-4735-b83d-c02813e30e09`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:25,809::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:21:25,809::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000582048', 'lastCheck': '0.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:21:25,810::task::1191::Storage.TaskManager.Task::(prepare) Task=`88ac2c13-86ca-4735-b83d-c02813e30e09`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000582048', 'lastCheck': '0.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:21:25,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`88ac2c13-86ca-4735-b83d-c02813e30e09`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:25,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:25,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:25,810::task::993::Storage.TaskManager.Task::(_decref) Task=`88ac2c13-86ca-4735-b83d-c02813e30e09`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:21:25,830::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:21:26,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:26,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131567 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:28,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:29,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132498 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:30,717::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:30,719::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:30,814::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:21:31,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:31,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133169 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:33,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:33,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132139 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:35,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:35,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131981 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:35,549::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:35,550::__init__::232::IOProcess::(_processLogs) (4864) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:21:35,550::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:21:35,568::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418111 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:35,570::__init__::232::IOProcess::(_processLogs) (4865) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:35,571::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:35,572::__init__::232::IOProcess::(_processLogs) (4866) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:35,572::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:35,573::__init__::232::IOProcess::(_processLogs) (4867) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:35,574::__init__::232::IOProcess::(_processLogs) (4868) Got request for method 'access' >Thread-86::DEBUG::2014-08-11 14:21:35,794::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:21:35,795::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec48f539-f53f-443c-bd4c-ee3d0e7f9786`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:35,813::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:35,821::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:35,823::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:35,824::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:21:35,826::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:35,827::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec48f539-f53f-443c-bd4c-ee3d0e7f9786`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:35,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec48f539-f53f-443c-bd4c-ee3d0e7f9786`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:35,827::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:35,827::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:35,828::task::993::Storage.TaskManager.Task::(_decref) Task=`ec48f539-f53f-443c-bd4c-ee3d0e7f9786`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:21:35,839::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:21:35,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`135c1da6-9873-4c14-a689-b1086245eaf6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:21:35,840::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:21:35,840::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8c984513-cfae-495c-a99d-63b74c680fdb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:21:35,841::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:21:35,841::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:21:35,841::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8c984513-cfae-495c-a99d-63b74c680fdb`::Granted request >Thread-34::DEBUG::2014-08-11 14:21:35,842::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`135c1da6-9873-4c14-a689-b1086245eaf6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:21:35,842::task::993::Storage.TaskManager.Task::(_decref) Task=`135c1da6-9873-4c14-a689-b1086245eaf6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:21:35,862::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:35,862::task::1191::Storage.TaskManager.Task::(prepare) Task=`135c1da6-9873-4c14-a689-b1086245eaf6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:35,862::task::595::Storage.TaskManager.Task::(_updateState) Task=`135c1da6-9873-4c14-a689-b1086245eaf6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:21:35,863::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:21:35,863::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:21:35,863::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:21:35,864::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:21:35,864::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:21:35,864::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:21:35,864::task::993::Storage.TaskManager.Task::(_decref) Task=`135c1da6-9873-4c14-a689-b1086245eaf6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:21:37,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:37,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132048 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:39,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:39,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131461 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:40,831::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:40,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:40,915::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:21:40,922::task::595::Storage.TaskManager.Task::(_updateState) Task=`1191fea1-d6fc-41b7-b36d-72b5ef95c645`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:40,923::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-86::INFO::2014-08-11 14:21:40,923::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418111', 'lastCheck': '5.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:21:40,923::task::1191::Storage.TaskManager.Task::(prepare) Task=`1191fea1-d6fc-41b7-b36d-72b5ef95c645`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418111', 'lastCheck': '5.3', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:21:40,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`1191fea1-d6fc-41b7-b36d-72b5ef95c645`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:40,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:40,924::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:40,924::task::993::Storage.TaskManager.Task::(_decref) Task=`1191fea1-d6fc-41b7-b36d-72b5ef95c645`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:21:40,944::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:21:41,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:41,345::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134814 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:43,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:43,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131997 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:45,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:45,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132422 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:45,576::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:45,576::__init__::232::IOProcess::(_processLogs) (4869) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:21:45,577::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:21:45,595::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415726 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:45,596::__init__::232::IOProcess::(_processLogs) (4870) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:45,597::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:45,598::__init__::232::IOProcess::(_processLogs) (4871) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:45,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:45,599::__init__::232::IOProcess::(_processLogs) (4872) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:45,600::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:45,600::__init__::232::IOProcess::(_processLogs) (4873) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:45,601::__init__::232::IOProcess::(_processLogs) (4873) Queuing response >Thread-86::DEBUG::2014-08-11 14:21:45,876::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:21:45,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eed04c8-e5c2-4a8c-9edb-e2aa03ad2869`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:45,913::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:45,922::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:45,923::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:45,924::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:21:45,927::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:45,927::task::1191::Storage.TaskManager.Task::(prepare) Task=`7eed04c8-e5c2-4a8c-9edb-e2aa03ad2869`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:45,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eed04c8-e5c2-4a8c-9edb-e2aa03ad2869`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:45,928::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:45,928::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:45,928::task::993::Storage.TaskManager.Task::(_decref) Task=`7eed04c8-e5c2-4a8c-9edb-e2aa03ad2869`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:21:45,938::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:21:45,938::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc99eb62-542c-41f5-a6a8-b99566fbed5e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:21:45,939::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:21:45,940::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9597ab1e-f16f-4875-a26d-11dd82f6191b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:21:45,940::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:21:45,940::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:21:45,940::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9597ab1e-f16f-4875-a26d-11dd82f6191b`::Granted request >Thread-34::DEBUG::2014-08-11 14:21:45,941::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dc99eb62-542c-41f5-a6a8-b99566fbed5e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:21:45,941::task::993::Storage.TaskManager.Task::(_decref) Task=`dc99eb62-542c-41f5-a6a8-b99566fbed5e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:21:45,961::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:45,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc99eb62-542c-41f5-a6a8-b99566fbed5e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:45,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc99eb62-542c-41f5-a6a8-b99566fbed5e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:21:45,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:21:45,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:21:45,962::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:21:45,962::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:21:45,963::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:21:45,963::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:21:45,963::task::993::Storage.TaskManager.Task::(_decref) Task=`dc99eb62-542c-41f5-a6a8-b99566fbed5e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:21:47,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:47,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133054 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:49,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:49,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013279 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:50,931::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:50,933::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:51,014::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:21:51,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:51,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143666 s, 71.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:53,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:53,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131783 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:55,603::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:55,604::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:21:55,605::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:21:55,605::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:55,623::__init__::232::IOProcess::(_processLogs) (4874) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:55,623::__init__::232::IOProcess::(_processLogs) (4874) Queuing response >Thread-16::DEBUG::2014-08-11 14:21:55,624::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000865844 s, 525 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:21:55,625::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:21:55,626::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:21:55,626::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:55,626::__init__::232::IOProcess::(_processLogs) (4875) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:21:55,627::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:55,628::__init__::232::IOProcess::(_processLogs) (4876) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:55,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:55,629::__init__::232::IOProcess::(_processLogs) (4877) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:21:55,629::__init__::232::IOProcess::(_processLogs) (4877) Queuing response >Thread-14::DEBUG::2014-08-11 14:21:55,630::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:21:55,631::__init__::232::IOProcess::(_processLogs) (4878) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:21:55,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:55,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131748 s, 77.7 MB/s\n'; <rc> = 0 >Thread-86::DEBUG::2014-08-11 14:21:55,977::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:21:55,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cd2bd2b-ba7e-4623-8077-0e46a3e0b336`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:56,014::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:56,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:56,023::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:21:56,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:21:56,027::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:56,027::task::1191::Storage.TaskManager.Task::(prepare) Task=`9cd2bd2b-ba7e-4623-8077-0e46a3e0b336`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:21:56,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cd2bd2b-ba7e-4623-8077-0e46a3e0b336`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:21:56,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:56,028::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:56,028::task::993::Storage.TaskManager.Task::(_decref) Task=`9cd2bd2b-ba7e-4623-8077-0e46a3e0b336`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:21:56,039::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:21:56,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ab5eea2-6107-46e2-ab54-98b2f1f23650`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:21:56,040::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:21:56,044::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b1d04163-1fd2-45a1-9548-13349a0eb37b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:21:56,045::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-86::DEBUG::2014-08-11 14:21:56,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`0955c0b1-a63b-4f0a-9003-ec34fe167703`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:21:56,047::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-34::DEBUG::2014-08-11 14:21:56,047::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:21:56,049::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b1d04163-1fd2-45a1-9548-13349a0eb37b`::Granted request >Thread-86::INFO::2014-08-11 14:21:56,049::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000865844', 'lastCheck': '0.4', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:21:56,051::task::1191::Storage.TaskManager.Task::(prepare) Task=`0955c0b1-a63b-4f0a-9003-ec34fe167703`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000865844', 'lastCheck': '0.4', 'valid': True}} >Thread-34::DEBUG::2014-08-11 14:21:56,052::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3ab5eea2-6107-46e2-ab54-98b2f1f23650`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-86::DEBUG::2014-08-11 14:21:56,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`0955c0b1-a63b-4f0a-9003-ec34fe167703`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:21:56,053::task::993::Storage.TaskManager.Task::(_decref) Task=`3ab5eea2-6107-46e2-ab54-98b2f1f23650`::ref 1 aborting False >Thread-86::DEBUG::2014-08-11 14:21:56,053::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:21:56,055::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:21:56,056::task::993::Storage.TaskManager.Task::(_decref) Task=`0955c0b1-a63b-4f0a-9003-ec34fe167703`::ref 0 aborting False >Thread-34::INFO::2014-08-11 14:21:56,098::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:21:56,099::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ab5eea2-6107-46e2-ab54-98b2f1f23650`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-86::DEBUG::2014-08-11 14:21:56,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:21:56,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ab5eea2-6107-46e2-ab54-98b2f1f23650`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:21:56,102::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:21:56,103::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:21:56,106::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:21:56,108::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:21:56,109::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:21:56,111::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:21:56,114::task::993::Storage.TaskManager.Task::(_decref) Task=`3ab5eea2-6107-46e2-ab54-98b2f1f23650`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:21:57,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:57,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013394 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:21:59,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:21:59,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215826 s, 47.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:01,027::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:01,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:01,115::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:01,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:01,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132667 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:03,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:03,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211115 s, 48.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:05,633::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:05,634::__init__::232::IOProcess::(_processLogs) (4879) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:22:05,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:22:05,653::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419889 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:05,654::__init__::232::IOProcess::(_processLogs) (4880) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:05,656::__init__::232::IOProcess::(_processLogs) (4881) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:05,657::__init__::232::IOProcess::(_processLogs) (4882) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:05,658::__init__::232::IOProcess::(_processLogs) (4883) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:22:06,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:06,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133225 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:06,122::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:06,124::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:06,125::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:22:06,131::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:22:06,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc982137-f289-4d2d-b263-b53eeb547e26`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:22:06,132::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:22:06,145::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:22:06,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc982137-f289-4d2d-b263-b53eeb547e26`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:22:06,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc982137-f289-4d2d-b263-b53eeb547e26`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:22:06,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:22:06,146::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:22:06,146::task::993::Storage.TaskManager.Task::(_decref) Task=`dc982137-f289-4d2d-b263-b53eeb547e26`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:22:06,156::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:22:06,157::task::595::Storage.TaskManager.Task::(_updateState) Task=`c16c4397-684b-4442-b4c2-702a2a9110c7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:22:06,157::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:22:06,158::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d280847a-75ca-415c-81b4-928438b772e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:22:06,158::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:22:06,158::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:22:06,159::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d280847a-75ca-415c-81b4-928438b772e4`::Granted request >Thread-34::DEBUG::2014-08-11 14:22:06,159::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c16c4397-684b-4442-b4c2-702a2a9110c7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:22:06,159::task::993::Storage.TaskManager.Task::(_decref) Task=`c16c4397-684b-4442-b4c2-702a2a9110c7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:22:06,179::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:06,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`c16c4397-684b-4442-b4c2-702a2a9110c7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:06,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`c16c4397-684b-4442-b4c2-702a2a9110c7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:22:06,180::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:22:06,180::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:22:06,181::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:22:06,181::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:22:06,181::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:22:06,181::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:22:06,182::task::993::Storage.TaskManager.Task::(_decref) Task=`c16c4397-684b-4442-b4c2-702a2a9110c7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:22:08,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:08,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131823 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:22:09,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf454955-5e87-491c-ad11-05b4d93eba07`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:22:09,637::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:22:09,638::__init__::232::IOProcess::(_processLogs) (4884) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:22:09,640::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:09,640::__init__::232::IOProcess::(_processLogs) (4885) Queuing response >Thread-55::INFO::2014-08-11 14:22:09,641::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:22:09,641::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf454955-5e87-491c-ad11-05b4d93eba07`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:22:09,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf454955-5e87-491c-ad11-05b4d93eba07`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:22:09,641::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:22:09,641::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:22:09,642::task::993::Storage.TaskManager.Task::(_decref) Task=`cf454955-5e87-491c-ad11-05b4d93eba07`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:22:10,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:10,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157576 s, 65.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:11,132::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:11,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:22:11,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`a12feb40-fa6b-493d-b56a-03d4cedcce60`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:22:11,217::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:11,219::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:22:11,219::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419889', 'lastCheck': '5.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:22:11,221::task::1191::Storage.TaskManager.Task::(prepare) Task=`a12feb40-fa6b-493d-b56a-03d4cedcce60`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419889', 'lastCheck': '5.6', 'valid': True}} >Thread-86::DEBUG::2014-08-11 14:22:11,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`a12feb40-fa6b-493d-b56a-03d4cedcce60`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:22:11,223::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:22:11,224::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:22:11,224::task::993::Storage.TaskManager.Task::(_decref) Task=`a12feb40-fa6b-493d-b56a-03d4cedcce60`::ref 0 aborting False >Thread-86::DEBUG::2014-08-11 14:22:11,243::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:12,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:12,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131689 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:14,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:14,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133236 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:15,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:15,660::__init__::232::IOProcess::(_processLogs) (4886) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:15,660::__init__::232::IOProcess::(_processLogs) (4886) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:15,661::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:22:15,679::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000410451 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:15,681::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:15,681::__init__::232::IOProcess::(_processLogs) (4887) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:15,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:15,682::__init__::232::IOProcess::(_processLogs) (4888) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:15,683::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:15,684::__init__::232::IOProcess::(_processLogs) (4889) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:15,684::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:15,685::__init__::232::IOProcess::(_processLogs) (4890) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:15,685::__init__::232::IOProcess::(_processLogs) (4890) Queuing response >Thread-86::DEBUG::2014-08-11 14:22:16,194::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:22:16,195::task::595::Storage.TaskManager.Task::(_updateState) Task=`67ccc974-241b-47fb-85bf-3141e595dec1`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:22:16,215::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-86::INFO::2014-08-11 14:22:16,228::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:22:16,229::task::1191::Storage.TaskManager.Task::(prepare) Task=`67ccc974-241b-47fb-85bf-3141e595dec1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:22:16,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`67ccc974-241b-47fb-85bf-3141e595dec1`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:16,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::DEBUG::2014-08-11 14:22:16,232::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:22:16,233::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:22:16,235::task::993::Storage.TaskManager.Task::(_decref) Task=`67ccc974-241b-47fb-85bf-3141e595dec1`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:16,236::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:16,239::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:22:16,248::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:22:16,249::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdb2a1ae-75ad-429f-af4b-324f450314e5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:22:16,249::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:22:16,250::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4c9636ff-dcbe-4bcf-8ec1-5f422dfa62fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:22:16,250::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:22:16,250::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:22:16,251::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4c9636ff-dcbe-4bcf-8ec1-5f422dfa62fa`::Granted request >Thread-34::DEBUG::2014-08-11 14:22:16,251::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bdb2a1ae-75ad-429f-af4b-324f450314e5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:22:16,251::task::993::Storage.TaskManager.Task::(_decref) Task=`bdb2a1ae-75ad-429f-af4b-324f450314e5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:22:16,271::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:16,272::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdb2a1ae-75ad-429f-af4b-324f450314e5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085522944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:16,272::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdb2a1ae-75ad-429f-af4b-324f450314e5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:22:16,272::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:22:16,272::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:22:16,273::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:22:16,273::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:22:16,273::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:22:16,273::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:22:16,274::task::993::Storage.TaskManager.Task::(_decref) Task=`bdb2a1ae-75ad-429f-af4b-324f450314e5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:22:16,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:16,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132837 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:18,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:18,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132084 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:20,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:20,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137839 s, 74.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:21,246::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:21,248::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:21,316::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:22,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:22,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131598 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:24,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:24,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132962 s, 77.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:22:25,686::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:22:25,688::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,689::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,689::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,689::__init__::232::IOProcess::(_processLogs) (4891) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:22:25,689::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|(4891) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:25,697::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:22:25,698::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,698::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,699::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,699::__init__::232::IOProcess::(_processLogs) (4892) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 14:22:25,702::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|(4892) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:25,702::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:22:25,703::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,703::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,704::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,704::__init__::232::IOProcess::(_processLogs) (4893) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:22:25,704::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|(4893) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:25,705::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:22:25,707::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,707::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,708::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,708::__init__::232::IOProcess::(_processLogs) (4894) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:25,708::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|(4894) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:25,709::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,710::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,710::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,710::__init__::232::IOProcess::(_processLogs) (4895) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:22:25,711::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 14:22:25,711::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|(4895) Queuing response >Thread-16::INFO::2014-08-11 14:22:25,712::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:22:25,713::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:22:25,713::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,713::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,714::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|(4896) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:25,714::__init__::232::IOProcess::(_processLogs) (4896) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:25,715::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:22:25,733::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418813 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:25,734::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,735::__init__::232::IOProcess::(_processLogs) (4890) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,735::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,735::__init__::232::IOProcess::(_processLogs) (4897) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:25,736::__init__::232::IOProcess::(_processLogs) DEBUG|(4897) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:25,737::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,738::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,739::__init__::232::IOProcess::(_processLogs) (4898) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:25,739::__init__::232::IOProcess::(_processLogs) (4898) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:25,739::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,740::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,740::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,741::__init__::232::IOProcess::(_processLogs) (4899) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:25,741::__init__::232::IOProcess::(_processLogs) (4899) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:25,741::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:25,742::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:25,743::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:25,743::__init__::232::IOProcess::(_processLogs) (4900) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:25,744::__init__::232::IOProcess::(_processLogs) (4900) Queuing response >Thread-86::DEBUG::2014-08-11 14:22:26,285::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-86::DEBUG::2014-08-11 14:22:26,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`111a463f-e333-48eb-9c4d-416d8052d83a`::moving from state init -> state preparing >Thread-86::INFO::2014-08-11 14:22:26,315::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:26,324::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:26,326::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:26,328::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-86::INFO::2014-08-11 14:22:26,329::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:22:26,331::task::1191::Storage.TaskManager.Task::(prepare) Task=`111a463f-e333-48eb-9c4d-416d8052d83a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-86::DEBUG::2014-08-11 14:22:26,331::task::595::Storage.TaskManager.Task::(_updateState) Task=`111a463f-e333-48eb-9c4d-416d8052d83a`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-08-11 14:22:26,332::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-08-11 14:22:26,332::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-08-11 14:22:26,332::task::993::Storage.TaskManager.Task::(_decref) Task=`111a463f-e333-48eb-9c4d-416d8052d83a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:22:26,342::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:22:26,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`2755093a-b37d-4d69-80a7-5973ad1a122c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:22:26,343::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:22:26,344::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d2029a2-8fe5-46b2-8c94-8420ce7b5e4d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:22:26,344::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:22:26,345::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:22:26,345::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d2029a2-8fe5-46b2-8c94-8420ce7b5e4d`::Granted request >Thread-34::DEBUG::2014-08-11 14:22:26,345::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2755093a-b37d-4d69-80a7-5973ad1a122c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:22:26,345::task::993::Storage.TaskManager.Task::(_decref) Task=`2755093a-b37d-4d69-80a7-5973ad1a122c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:22:26,365::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:26,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`2755093a-b37d-4d69-80a7-5973ad1a122c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:26,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`2755093a-b37d-4d69-80a7-5973ad1a122c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:22:26,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:22:26,367::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:22:26,367::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:22:26,367::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:22:26,367::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:22:26,368::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:22:26,368::task::993::Storage.TaskManager.Task::(_decref) Task=`2755093a-b37d-4d69-80a7-5973ad1a122c`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 14:22:26,549::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:54608 >Detector thread::DEBUG::2014-08-11 14:22:26,556::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:54608 >Detector thread::DEBUG::2014-08-11 14:22:26,558::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:54608 >Detector thread::DEBUG::2014-08-11 14:22:26,559::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 54608) >Dummy-33::DEBUG::2014-08-11 14:22:26,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-89::DEBUG::2014-08-11 14:22:26,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8bd9f1c-18d5-4ecf-ba21-f0e0d0a03ebb`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:22:26,588::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:22:26,589::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418813', 'lastCheck': '0.8', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:22:26,590::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8bd9f1c-18d5-4ecf-ba21-f0e0d0a03ebb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418813', 'lastCheck': '0.8', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:22:26,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8bd9f1c-18d5-4ecf-ba21-f0e0d0a03ebb`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:22:26,592::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:22:26,592::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:22:26,593::task::993::Storage.TaskManager.Task::(_decref) Task=`c8bd9f1c-18d5-4ecf-ba21-f0e0d0a03ebb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:22:26,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.037778 s, 27.1 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:22:26,671::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:28,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:28,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133212 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:30,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:30,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131775 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:31,333::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:31,335::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:31,417::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:32,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:32,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0291663 s, 35.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:34,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:34,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131927 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:35,744::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:35,744::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:35,745::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:35,745::__init__::232::IOProcess::(_processLogs) (4901) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:35,746::__init__::232::IOProcess::(_processLogs) DEBUG|(4901) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:35,746::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:22:35,766::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00168253 s, 270 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:35,767::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:35,767::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:35,768::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:35,768::__init__::232::IOProcess::(_processLogs) (4902) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:35,769::__init__::232::IOProcess::(_processLogs) DEBUG|(4902) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:35,770::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:35,770::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:35,771::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:35,771::__init__::232::IOProcess::(_processLogs) (4903) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:35,771::__init__::232::IOProcess::(_processLogs) DEBUG|(4903) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:35,772::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:35,773::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:35,773::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:35,773::__init__::232::IOProcess::(_processLogs) (4904) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:35,774::__init__::232::IOProcess::(_processLogs) DEBUG|(4904) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:35,775::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:35,775::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:35,775::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:35,776::__init__::232::IOProcess::(_processLogs) (4905) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:35,776::__init__::232::IOProcess::(_processLogs) DEBUG|(4905) Queuing response >Thread-89::DEBUG::2014-08-11 14:22:36,380::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:22:36,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`6278ce91-6b4a-4281-a381-18a53ea3fa30`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:22:36,416::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:36,425::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:36,426::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:36,427::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:22:36,429::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:22:36,430::task::1191::Storage.TaskManager.Task::(prepare) Task=`6278ce91-6b4a-4281-a381-18a53ea3fa30`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:22:36,430::task::595::Storage.TaskManager.Task::(_updateState) Task=`6278ce91-6b4a-4281-a381-18a53ea3fa30`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:22:36,430::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:22:36,431::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:22:36,431::task::993::Storage.TaskManager.Task::(_decref) Task=`6278ce91-6b4a-4281-a381-18a53ea3fa30`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:22:36,444::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:22:36,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8759e13-62ff-43ee-844b-fd9a36f3459d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:22:36,445::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:22:36,445::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`59cbf3ff-f6aa-402f-ad96-a6a05ae4c0a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:22:36,446::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:22:36,446::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:22:36,446::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`59cbf3ff-f6aa-402f-ad96-a6a05ae4c0a7`::Granted request >Thread-34::DEBUG::2014-08-11 14:22:36,447::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b8759e13-62ff-43ee-844b-fd9a36f3459d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:22:36,447::task::993::Storage.TaskManager.Task::(_decref) Task=`b8759e13-62ff-43ee-844b-fd9a36f3459d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:22:36,467::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:36,467::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8759e13-62ff-43ee-844b-fd9a36f3459d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:36,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8759e13-62ff-43ee-844b-fd9a36f3459d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:22:36,468::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:22:36,468::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:22:36,468::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:22:36,469::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:22:36,469::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:22:36,469::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:22:36,469::task::993::Storage.TaskManager.Task::(_decref) Task=`b8759e13-62ff-43ee-844b-fd9a36f3459d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:22:37,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:37,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132153 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:39,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:39,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131808 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:41,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:41,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132702 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:41,430::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:41,431::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:41,517::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:22:41,789::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad09b6e8-c7c5-4955-a1d4-55ad18b3e7a2`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:22:41,790::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:22:41,790::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00168253', 'lastCheck': '6.0', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:22:41,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad09b6e8-c7c5-4955-a1d4-55ad18b3e7a2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00168253', 'lastCheck': '6.0', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:22:41,791::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad09b6e8-c7c5-4955-a1d4-55ad18b3e7a2`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:22:41,791::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:22:41,791::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:22:41,791::task::993::Storage.TaskManager.Task::(_decref) Task=`ad09b6e8-c7c5-4955-a1d4-55ad18b3e7a2`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:22:41,810::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:43,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:43,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132788 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:45,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:45,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132514 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:45,778::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:45,778::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:45,779::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:45,779::__init__::232::IOProcess::(_processLogs) (4906) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:45,780::__init__::232::IOProcess::(_processLogs) DEBUG|(4906) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:45,781::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:22:45,799::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000434904 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:45,800::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:45,801::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:45,801::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:45,801::__init__::232::IOProcess::(_processLogs) (4907) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:45,802::__init__::232::IOProcess::(_processLogs) DEBUG|(4907) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:45,803::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:45,804::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:45,804::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:45,804::__init__::232::IOProcess::(_processLogs) (4908) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:45,805::__init__::232::IOProcess::(_processLogs) DEBUG|(4908) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:45,806::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:45,806::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:45,806::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:45,806::__init__::232::IOProcess::(_processLogs) (4909) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:45,807::__init__::232::IOProcess::(_processLogs) DEBUG|(4909) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:45,808::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:45,808::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:45,808::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:45,809::__init__::232::IOProcess::(_processLogs) (4910) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:45,809::__init__::232::IOProcess::(_processLogs) DEBUG|(4910) Queuing response >Thread-89::DEBUG::2014-08-11 14:22:46,481::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:22:46,482::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8ce9b1f-fcb0-4494-bc79-768743053635`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:22:46,516::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:46,524::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:46,526::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:46,527::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:22:46,530::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:22:46,530::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8ce9b1f-fcb0-4494-bc79-768743053635`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:22:46,530::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8ce9b1f-fcb0-4494-bc79-768743053635`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:22:46,531::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:22:46,531::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:22:46,531::task::993::Storage.TaskManager.Task::(_decref) Task=`c8ce9b1f-fcb0-4494-bc79-768743053635`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:22:46,542::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:22:46,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`aeebf758-66ae-4c2b-a1e5-241a373901e4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:22:46,543::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:22:46,543::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0259cc2-7935-4e87-9343-29bc0a0fe1b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:22:46,543::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:22:46,544::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:22:46,544::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0259cc2-7935-4e87-9343-29bc0a0fe1b5`::Granted request >Thread-34::DEBUG::2014-08-11 14:22:46,545::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aeebf758-66ae-4c2b-a1e5-241a373901e4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:22:46,545::task::993::Storage.TaskManager.Task::(_decref) Task=`aeebf758-66ae-4c2b-a1e5-241a373901e4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:22:46,565::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:46,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`aeebf758-66ae-4c2b-a1e5-241a373901e4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:46,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`aeebf758-66ae-4c2b-a1e5-241a373901e4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:22:46,565::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:22:46,566::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:22:46,566::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:22:46,566::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:22:46,566::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:22:46,567::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:22:46,567::task::993::Storage.TaskManager.Task::(_decref) Task=`aeebf758-66ae-4c2b-a1e5-241a373901e4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:22:47,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:47,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131115 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:49,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:49,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132026 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:51,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:51,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132545 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:51,532::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:51,534::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:51,617::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:53,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:53,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132681 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:55,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:55,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013246 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:55,811::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:55,811::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:55,812::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:55,812::__init__::232::IOProcess::(_processLogs) (4911) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:55,813::__init__::232::IOProcess::(_processLogs) DEBUG|(4911) Queuing response >Thread-16::DEBUG::2014-08-11 14:22:55,814::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:22:55,833::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419908 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:22:55,834::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:55,835::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:55,835::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:55,835::__init__::232::IOProcess::(_processLogs) (4912) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:22:55,836::__init__::232::IOProcess::(_processLogs) DEBUG|(4912) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:55,837::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:55,837::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:55,838::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:55,838::__init__::232::IOProcess::(_processLogs) (4913) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:55,839::__init__::232::IOProcess::(_processLogs) DEBUG|(4913) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:55,840::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:55,840::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:55,840::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:55,841::__init__::232::IOProcess::(_processLogs) (4914) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:55,841::__init__::232::IOProcess::(_processLogs) DEBUG|(4914) Queuing response >Thread-14::DEBUG::2014-08-11 14:22:55,842::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:22:55,843::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:22:55,843::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:22:55,843::__init__::232::IOProcess::(_processLogs) (4915) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:22:55,844::__init__::232::IOProcess::(_processLogs) DEBUG|(4915) Queuing response >Thread-89::DEBUG::2014-08-11 14:22:56,579::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:22:56,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b15e84f-6ea6-4474-986e-6205ec12b40d`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:22:56,617::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:56,624::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:56,626::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:22:56,627::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:22:56,629::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:22:56,629::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b15e84f-6ea6-4474-986e-6205ec12b40d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:22:56,630::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b15e84f-6ea6-4474-986e-6205ec12b40d`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:22:56,630::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:22:56,630::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:22:56,630::task::993::Storage.TaskManager.Task::(_decref) Task=`0b15e84f-6ea6-4474-986e-6205ec12b40d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:22:56,641::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:22:56,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`6352e084-2d7e-4b17-ac7e-e334343cb4ba`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:22:56,642::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:22:56,642::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`45fd9dde-1f6b-45d6-8663-c053a8dee00d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:22:56,642::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:22:56,643::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:22:56,643::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`45fd9dde-1f6b-45d6-8663-c053a8dee00d`::Granted request >Thread-34::DEBUG::2014-08-11 14:22:56,644::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6352e084-2d7e-4b17-ac7e-e334343cb4ba`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:22:56,644::task::993::Storage.TaskManager.Task::(_decref) Task=`6352e084-2d7e-4b17-ac7e-e334343cb4ba`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:22:56,664::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:56,664::task::1191::Storage.TaskManager.Task::(prepare) Task=`6352e084-2d7e-4b17-ac7e-e334343cb4ba`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085457408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:22:56,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`6352e084-2d7e-4b17-ac7e-e334343cb4ba`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:22:56,664::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:22:56,665::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:22:56,665::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:22:56,665::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:22:56,666::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:22:56,666::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:22:56,666::task::993::Storage.TaskManager.Task::(_decref) Task=`6352e084-2d7e-4b17-ac7e-e334343cb4ba`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:22:56,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ffcb5cd-17c7-4753-8e39-906b030cb862`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:22:56,899::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:22:56,899::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419908', 'lastCheck': '1.1', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:22:56,900::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ffcb5cd-17c7-4753-8e39-906b030cb862`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419908', 'lastCheck': '1.1', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:22:56,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ffcb5cd-17c7-4753-8e39-906b030cb862`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:22:56,900::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:22:56,900::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:22:56,900::task::993::Storage.TaskManager.Task::(_decref) Task=`4ffcb5cd-17c7-4753-8e39-906b030cb862`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:22:56,919::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:22:57,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:57,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131595 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:22:59,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:22:59,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132571 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:01,634::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:01,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:01,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:01,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:01,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013429 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:03,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:03,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133288 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:05,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:23:05,847::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:05,848::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:05,848::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:05,849::__init__::232::IOProcess::(_processLogs) DEBUG|(4916) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:05,849::__init__::232::IOProcess::(_processLogs) (4916) Queuing response >Thread-16::DEBUG::2014-08-11 14:23:05,850::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:05,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243093 s, 42.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:23:05,876::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00179645 s, 253 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:05,878::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:05,880::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:05,881::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:05,882::__init__::232::IOProcess::(_processLogs) (4917) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:05,884::__init__::232::IOProcess::(_processLogs) DEBUG|(4917) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:05,887::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:05,888::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:05,889::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:05,891::__init__::232::IOProcess::(_processLogs) (4918) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:05,893::__init__::232::IOProcess::(_processLogs) DEBUG|(4918) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:05,895::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:05,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:05,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:05,899::__init__::232::IOProcess::(_processLogs) (4919) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:05,901::__init__::232::IOProcess::(_processLogs) (4919) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:05,904::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:05,905::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:05,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:05,908::__init__::232::IOProcess::(_processLogs) (4920) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:05,910::__init__::232::IOProcess::(_processLogs) DEBUG|(4920) Queuing response >Thread-89::DEBUG::2014-08-11 14:23:06,678::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:23:06,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`56372354-e5d5-42d6-9164-1a1536c54c84`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:06,716::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:23:06,729::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:06,729::task::1191::Storage.TaskManager.Task::(prepare) Task=`56372354-e5d5-42d6-9164-1a1536c54c84`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:06,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`56372354-e5d5-42d6-9164-1a1536c54c84`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:06,730::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:06,730::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:06,730::task::993::Storage.TaskManager.Task::(_decref) Task=`56372354-e5d5-42d6-9164-1a1536c54c84`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:23:06,741::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:23:06,742::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a20dc15-e458-4f5e-afde-9f4e20504710`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:23:06,742::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:23:06,745::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2c64c24d-bbc4-4cc5-816c-5b148ee0ff1c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:06,745::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:23:06,746::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:23:06,748::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:23:06,749::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2c64c24d-bbc4-4cc5-816c-5b148ee0ff1c`::Granted request >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:06,751::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:23:06,753::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8a20dc15-e458-4f5e-afde-9f4e20504710`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:23:06,755::task::993::Storage.TaskManager.Task::(_decref) Task=`8a20dc15-e458-4f5e-afde-9f4e20504710`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:06,756::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:23:06,779::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:06,779::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a20dc15-e458-4f5e-afde-9f4e20504710`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:06,779::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a20dc15-e458-4f5e-afde-9f4e20504710`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:23:06,780::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:23:06,780::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:23:06,780::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:23:06,781::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:23:06,781::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:23:06,781::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:23:06,781::task::993::Storage.TaskManager.Task::(_decref) Task=`8a20dc15-e458-4f5e-afde-9f4e20504710`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:23:07,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:07,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131774 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:23:09,745::task::595::Storage.TaskManager.Task::(_updateState) Task=`2458c705-e0e9-4e5d-b656-c39c099c9e2e`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:23:09,746::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:23:09,747::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:09,747::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:09,747::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:09,748::__init__::232::IOProcess::(_processLogs) (4921) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:23:09,748::__init__::232::IOProcess::(_processLogs) DEBUG|(4921) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:09,749::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:09,749::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:09,750::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:09,750::__init__::232::IOProcess::(_processLogs) (4922) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:23:09,750::__init__::232::IOProcess::(_processLogs) (4922) Queuing response >Thread-55::INFO::2014-08-11 14:23:09,751::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:23:09,751::task::1191::Storage.TaskManager.Task::(prepare) Task=`2458c705-e0e9-4e5d-b656-c39c099c9e2e`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:23:09,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`2458c705-e0e9-4e5d-b656-c39c099c9e2e`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:23:09,751::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:23:09,752::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:23:09,752::task::993::Storage.TaskManager.Task::(_decref) Task=`2458c705-e0e9-4e5d-b656-c39c099c9e2e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:23:09,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:10,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131843 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:11,766::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:11,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:11,817::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:23:12,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`06723ee9-a9b3-4877-8064-aa11376d44d0`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:12,006::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:23:12,006::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00179645', 'lastCheck': '6.1', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:12,006::task::1191::Storage.TaskManager.Task::(prepare) Task=`06723ee9-a9b3-4877-8064-aa11376d44d0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00179645', 'lastCheck': '6.1', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:12,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`06723ee9-a9b3-4877-8064-aa11376d44d0`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:12,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:12,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:12,007::task::993::Storage.TaskManager.Task::(_decref) Task=`06723ee9-a9b3-4877-8064-aa11376d44d0`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:23:12,026::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:12,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:12,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132567 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:14,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:14,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132114 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:15,912::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:15,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:15,913::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:15,913::__init__::232::IOProcess::(_processLogs) (4923) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:15,913::__init__::232::IOProcess::(_processLogs) (4923) Queuing response >Thread-16::DEBUG::2014-08-11 14:23:15,914::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:23:15,932::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427685 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:15,934::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:15,934::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:15,934::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:15,935::__init__::232::IOProcess::(_processLogs) (4924) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:15,935::__init__::232::IOProcess::(_processLogs) DEBUG|(4924) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:15,937::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:15,937::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:15,940::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:15,940::__init__::232::IOProcess::(_processLogs) (4925) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:15,940::__init__::232::IOProcess::(_processLogs) (4925) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:15,941::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:15,941::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:15,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:15,942::__init__::232::IOProcess::(_processLogs) (4926) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:15,942::__init__::232::IOProcess::(_processLogs) DEBUG|(4926) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:15,943::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:15,944::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:15,944::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:15,945::__init__::232::IOProcess::(_processLogs) (4927) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:15,945::__init__::232::IOProcess::(_processLogs) (4927) Queuing response >Dummy-33::DEBUG::2014-08-11 14:23:16,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:16,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132412 s, 77.3 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:23:16,794::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:23:16,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`93cdfa70-fac9-4b61-8226-5945270b7d8e`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:16,817::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:16,824::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:16,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:16,827::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:23:16,830::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:16,830::task::1191::Storage.TaskManager.Task::(prepare) Task=`93cdfa70-fac9-4b61-8226-5945270b7d8e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:16,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`93cdfa70-fac9-4b61-8226-5945270b7d8e`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:16,831::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:16,831::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:16,831::task::993::Storage.TaskManager.Task::(_decref) Task=`93cdfa70-fac9-4b61-8226-5945270b7d8e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:23:16,844::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:23:16,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`42e07880-f5ae-44c5-99eb-b9de20579b6b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:23:16,845::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:23:16,846::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d7941d2c-0607-47c9-a9b8-bcb893959bb2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:23:16,846::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:23:16,847::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:23:16,847::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d7941d2c-0607-47c9-a9b8-bcb893959bb2`::Granted request >Thread-34::DEBUG::2014-08-11 14:23:16,847::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`42e07880-f5ae-44c5-99eb-b9de20579b6b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:23:16,848::task::993::Storage.TaskManager.Task::(_decref) Task=`42e07880-f5ae-44c5-99eb-b9de20579b6b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:23:16,867::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:16,868::task::1191::Storage.TaskManager.Task::(prepare) Task=`42e07880-f5ae-44c5-99eb-b9de20579b6b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:16,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`42e07880-f5ae-44c5-99eb-b9de20579b6b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:23:16,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:23:16,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:23:16,869::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:23:16,869::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:23:16,869::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:23:16,869::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:23:16,870::task::993::Storage.TaskManager.Task::(_decref) Task=`42e07880-f5ae-44c5-99eb-b9de20579b6b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:23:18,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:18,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132489 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:20,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:20,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132976 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:21,834::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:21,835::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:21,918::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:22,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:22,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132539 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:24,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:24,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013161 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:25,946::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:25,947::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:25,947::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:25,949::__init__::232::IOProcess::(_processLogs) (4928) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:23:25,949::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:23:25,950::__init__::232::IOProcess::(_processLogs) (4928) Queuing response >Thread-16::DEBUG::2014-08-11 14:23:25,969::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421981 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:25,970::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:25,971::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:25,971::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:25,971::__init__::232::IOProcess::(_processLogs) (4929) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:25,972::__init__::232::IOProcess::(_processLogs) DEBUG|(4929) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:25,973::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:25,974::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:25,974::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:25,975::__init__::232::IOProcess::(_processLogs) (4930) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:25,975::__init__::232::IOProcess::(_processLogs) (4930) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:25,975::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:25,976::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:25,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:25,977::__init__::232::IOProcess::(_processLogs) (4931) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:25,977::__init__::232::IOProcess::(_processLogs) (4931) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:25,977::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:25,978::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:25,978::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:25,979::__init__::232::IOProcess::(_processLogs) (4932) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:25,980::__init__::232::IOProcess::(_processLogs) (4932) Queuing response >Dummy-33::DEBUG::2014-08-11 14:23:26,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:26,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133039 s, 77.0 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:23:26,881::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:23:26,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c1a47b6-284c-46d9-b254-5783adf75066`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:26,917::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:26,926::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:26,927::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:26,928::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:23:26,930::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:26,930::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c1a47b6-284c-46d9-b254-5783adf75066`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:26,931::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c1a47b6-284c-46d9-b254-5783adf75066`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:26,931::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:26,931::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:26,931::task::993::Storage.TaskManager.Task::(_decref) Task=`2c1a47b6-284c-46d9-b254-5783adf75066`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:23:26,941::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:23:26,942::task::595::Storage.TaskManager.Task::(_updateState) Task=`8142f76d-5c4b-4a07-88b2-471ea655a1f0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:23:26,942::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:23:26,943::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d20a06a9-0f55-4026-b45e-412844308b4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:23:26,943::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:23:26,944::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:23:26,944::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d20a06a9-0f55-4026-b45e-412844308b4a`::Granted request >Thread-34::DEBUG::2014-08-11 14:23:26,945::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8142f76d-5c4b-4a07-88b2-471ea655a1f0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:23:26,945::task::993::Storage.TaskManager.Task::(_decref) Task=`8142f76d-5c4b-4a07-88b2-471ea655a1f0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:23:26,965::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:26,965::task::1191::Storage.TaskManager.Task::(prepare) Task=`8142f76d-5c4b-4a07-88b2-471ea655a1f0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:26,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`8142f76d-5c4b-4a07-88b2-471ea655a1f0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:23:26,966::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:23:26,966::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:23:26,966::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:23:26,966::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:23:26,967::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:23:26,967::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:23:26,967::task::993::Storage.TaskManager.Task::(_decref) Task=`8142f76d-5c4b-4a07-88b2-471ea655a1f0`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:23:27,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ef5c8ba-eded-4464-bf9a-feee5238669e`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:27,121::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:23:27,123::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421981', 'lastCheck': '1.1', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:27,125::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ef5c8ba-eded-4464-bf9a-feee5238669e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421981', 'lastCheck': '1.1', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:27,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ef5c8ba-eded-4464-bf9a-feee5238669e`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:27,127::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:27,128::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:27,129::task::993::Storage.TaskManager.Task::(_decref) Task=`1ef5c8ba-eded-4464-bf9a-feee5238669e`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:23:27,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:28,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:28,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132541 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:30,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:30,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132138 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:31,931::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:31,933::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:32,022::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:32,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:32,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269704 s, 38.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:34,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:34,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132444 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:35,980::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:35,981::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:23:35,982::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:23:35,983::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:36,001::__init__::232::IOProcess::(_processLogs) (4933) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:36,001::__init__::232::IOProcess::(_processLogs) (4933) Queuing response >Thread-16::DEBUG::2014-08-11 14:23:36,002::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000537528 s, 846 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:36,003::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:36,003::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:36,004::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:36,004::__init__::232::IOProcess::(_processLogs) (4934) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:36,005::__init__::232::IOProcess::(_processLogs) DEBUG|(4934) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:36,006::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:36,007::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:36,007::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:36,008::__init__::232::IOProcess::(_processLogs) (4935) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:36,008::__init__::232::IOProcess::(_processLogs) (4935) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:36,008::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:36,009::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:36,009::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:36,010::__init__::232::IOProcess::(_processLogs) (4936) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:36,010::__init__::232::IOProcess::(_processLogs) (4936) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:36,010::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:36,011::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:36,011::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:36,012::__init__::232::IOProcess::(_processLogs) (4937) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:36,013::__init__::232::IOProcess::(_processLogs) (4937) Queuing response >Dummy-33::DEBUG::2014-08-11 14:23:36,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-89::DEBUG::2014-08-11 14:23:36,996::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Dummy-33::DEBUG::2014-08-11 14:23:37,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013318 s, 76.9 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:23:37,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9c6c644-27c4-4c9e-b0d2-a365d03a94c3`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:37,021::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:37,033::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:37,036::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:37,039::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:23:37,045::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:37,047::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9c6c644-27c4-4c9e-b0d2-a365d03a94c3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:37,048::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9c6c644-27c4-4c9e-b0d2-a365d03a94c3`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:37,049::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:37,049::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:37,051::task::993::Storage.TaskManager.Task::(_decref) Task=`d9c6c644-27c4-4c9e-b0d2-a365d03a94c3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:23:37,063::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:23:37,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`f943d30c-c314-408e-8e63-21c29615f75e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:23:37,064::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:23:37,065::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6279edea-a01b-4bbc-8279-9ca651da25fb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:23:37,065::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:23:37,066::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:23:37,066::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6279edea-a01b-4bbc-8279-9ca651da25fb`::Granted request >Thread-34::DEBUG::2014-08-11 14:23:37,066::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f943d30c-c314-408e-8e63-21c29615f75e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:23:37,067::task::993::Storage.TaskManager.Task::(_decref) Task=`f943d30c-c314-408e-8e63-21c29615f75e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:23:37,086::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:37,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`f943d30c-c314-408e-8e63-21c29615f75e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:37,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`f943d30c-c314-408e-8e63-21c29615f75e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:23:37,087::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:23:37,087::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:23:37,088::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:23:37,088::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:23:37,088::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:23:37,089::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:23:37,089::task::993::Storage.TaskManager.Task::(_decref) Task=`f943d30c-c314-408e-8e63-21c29615f75e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:23:39,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:39,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131623 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:41,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:41,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141187 s, 72.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:42,048::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:42,049::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:42,050::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:23:42,314::task::595::Storage.TaskManager.Task::(_updateState) Task=`793c9391-b1b6-461b-8f4a-2958fdd519b4`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:42,314::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:23:42,315::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537528', 'lastCheck': '6.3', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:42,315::task::1191::Storage.TaskManager.Task::(prepare) Task=`793c9391-b1b6-461b-8f4a-2958fdd519b4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000537528', 'lastCheck': '6.3', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:42,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`793c9391-b1b6-461b-8f4a-2958fdd519b4`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:42,316::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:42,316::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:42,316::task::993::Storage.TaskManager.Task::(_decref) Task=`793c9391-b1b6-461b-8f4a-2958fdd519b4`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:23:42,336::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:43,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:43,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131426 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:45,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:45,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132107 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:46,013::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:46,014::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:46,015::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:23:46,016::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:23:46,017::__init__::232::IOProcess::(_processLogs) (4938) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:46,035::__init__::232::IOProcess::(_processLogs) (4938) Queuing response >Thread-16::DEBUG::2014-08-11 14:23:46,036::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000544916 s, 835 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:46,037::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:23:46,037::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:46,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:46,038::__init__::232::IOProcess::(_processLogs) (4939) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:46,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:23:46,039::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:46,040::__init__::232::IOProcess::(_processLogs) (4940) Queuing response >Thread-14::DEBUG::2014-08-11 14:23:46,041::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:46,041::__init__::232::IOProcess::(_processLogs) (4941) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:46,042::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:46,042::__init__::232::IOProcess::(_processLogs) (4942) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:47,057::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:47,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:23:47,102::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:23:47,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`f51137d0-66bc-4e9a-83a5-e6b25a19e36d`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:47,122::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:47,122::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:23:47,136::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:47,137::task::1191::Storage.TaskManager.Task::(prepare) Task=`f51137d0-66bc-4e9a-83a5-e6b25a19e36d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:47,137::task::595::Storage.TaskManager.Task::(_updateState) Task=`f51137d0-66bc-4e9a-83a5-e6b25a19e36d`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:47,137::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:47,137::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:47,137::task::993::Storage.TaskManager.Task::(_decref) Task=`f51137d0-66bc-4e9a-83a5-e6b25a19e36d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:23:47,150::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:23:47,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`77e9535a-16b1-4501-b215-ed1527d08534`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:23:47,151::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:23:47,151::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e52f9a46-5871-4cd2-8f9a-74677a2e1cd3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:23:47,152::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:23:47,152::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:23:47,152::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e52f9a46-5871-4cd2-8f9a-74677a2e1cd3`::Granted request >Thread-34::DEBUG::2014-08-11 14:23:47,153::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`77e9535a-16b1-4501-b215-ed1527d08534`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:23:47,153::task::993::Storage.TaskManager.Task::(_decref) Task=`77e9535a-16b1-4501-b215-ed1527d08534`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:23:47,173::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:47,173::task::1191::Storage.TaskManager.Task::(prepare) Task=`77e9535a-16b1-4501-b215-ed1527d08534`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085391872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:47,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`77e9535a-16b1-4501-b215-ed1527d08534`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:23:47,174::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:23:47,174::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:23:47,175::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:23:47,175::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:23:47,175::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:23:47,175::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:23:47,176::task::993::Storage.TaskManager.Task::(_decref) Task=`77e9535a-16b1-4501-b215-ed1527d08534`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:23:47,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:47,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0257364 s, 39.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:49,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:49,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013234 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:51,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:51,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132696 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:52,130::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:52,131::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:52,218::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:53,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:53,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147078 s, 69.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:55,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:55,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131849 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:56,044::__init__::232::IOProcess::(_processLogs) (4943) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:23:56,045::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:23:56,064::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415123 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:23:56,065::__init__::232::IOProcess::(_processLogs) (4944) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:23:56,066::__init__::232::IOProcess::(_processLogs) (4945) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:56,067::__init__::232::IOProcess::(_processLogs) (4946) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:23:56,068::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:23:56,068::__init__::232::IOProcess::(_processLogs) (4947) Got request for method 'access' >Thread-89::DEBUG::2014-08-11 14:23:57,187::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:23:57,188::task::595::Storage.TaskManager.Task::(_updateState) Task=`d090099a-2086-4596-b1f7-4d6bd2a344f9`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:57,218::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:57,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:57,227::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:23:57,228::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:23:57,230::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:57,230::task::1191::Storage.TaskManager.Task::(prepare) Task=`d090099a-2086-4596-b1f7-4d6bd2a344f9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:23:57,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`d090099a-2086-4596-b1f7-4d6bd2a344f9`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:57,231::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:57,231::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:57,231::task::993::Storage.TaskManager.Task::(_decref) Task=`d090099a-2086-4596-b1f7-4d6bd2a344f9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:23:57,241::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:23:57,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`2adbfe45-5b03-4dc4-ae2c-7ee236c44815`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:23:57,242::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:23:57,243::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1e61b992-3e60-4eb9-8d3c-496a3e930ba3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:23:57,243::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:23:57,244::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:23:57,244::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1e61b992-3e60-4eb9-8d3c-496a3e930ba3`::Granted request >Thread-34::DEBUG::2014-08-11 14:23:57,245::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2adbfe45-5b03-4dc4-ae2c-7ee236c44815`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:23:57,245::task::993::Storage.TaskManager.Task::(_decref) Task=`2adbfe45-5b03-4dc4-ae2c-7ee236c44815`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:23:57,264::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:57,265::task::1191::Storage.TaskManager.Task::(prepare) Task=`2adbfe45-5b03-4dc4-ae2c-7ee236c44815`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:23:57,265::task::595::Storage.TaskManager.Task::(_updateState) Task=`2adbfe45-5b03-4dc4-ae2c-7ee236c44815`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:23:57,265::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:23:57,266::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:23:57,266::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:23:57,266::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:23:57,266::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:23:57,267::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:23:57,267::task::993::Storage.TaskManager.Task::(_decref) Task=`2adbfe45-5b03-4dc4-ae2c-7ee236c44815`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:23:57,421::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb58b55f-f270-4da4-b51b-8931a77eb9e3`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:23:57,421::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:23:57,421::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415123', 'lastCheck': '1.4', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:57,422::task::1191::Storage.TaskManager.Task::(prepare) Task=`cb58b55f-f270-4da4-b51b-8931a77eb9e3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415123', 'lastCheck': '1.4', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:23:57,422::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb58b55f-f270-4da4-b51b-8931a77eb9e3`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:23:57,422::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:23:57,422::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:23:57,423::task::993::Storage.TaskManager.Task::(_decref) Task=`cb58b55f-f270-4da4-b51b-8931a77eb9e3`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:23:57,442::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:23:57,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:57,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131316 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:23:59,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:23:59,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013238 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:01,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:01,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132994 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:02,236::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:02,242::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:02,322::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:03,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:03,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132086 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:05,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:06,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013263 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:06,071::__init__::232::IOProcess::(_processLogs) (4948) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:24:06,072::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:24:06,090::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418183 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:06,091::__init__::232::IOProcess::(_processLogs) (4949) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:06,092::__init__::232::IOProcess::(_processLogs) (4950) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:06,094::__init__::232::IOProcess::(_processLogs) (4951) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:06,095::__init__::232::IOProcess::(_processLogs) (4952) Got request for method 'access' >Thread-89::DEBUG::2014-08-11 14:24:07,278::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-89::DEBUG::2014-08-11 14:24:07,279::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e2f5673-9444-4052-946b-0e952ef37b7e`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:07,317::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:24:07,330::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:07,332::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e2f5673-9444-4052-946b-0e952ef37b7e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:07,333::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:24:07,333::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e2f5673-9444-4052-946b-0e952ef37b7e`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:07,335::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:07,336::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:07,338::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:24:07,338::task::993::Storage.TaskManager.Task::(_decref) Task=`9e2f5673-9444-4052-946b-0e952ef37b7e`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:07,341::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:24:07,351::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 14:24:07,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`732fff57-52bc-4a0d-9e46-32a2ef53d559`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:24:07,352::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:24:07,352::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`39d609b1-4b96-434c-bdcc-7f0c9fa1440e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:24:07,352::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:24:07,353::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:24:07,353::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`39d609b1-4b96-434c-bdcc-7f0c9fa1440e`::Granted request >Thread-34::DEBUG::2014-08-11 14:24:07,354::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`732fff57-52bc-4a0d-9e46-32a2ef53d559`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:24:07,354::task::993::Storage.TaskManager.Task::(_decref) Task=`732fff57-52bc-4a0d-9e46-32a2ef53d559`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:24:07,374::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:07,374::task::1191::Storage.TaskManager.Task::(prepare) Task=`732fff57-52bc-4a0d-9e46-32a2ef53d559`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:07,374::task::595::Storage.TaskManager.Task::(_updateState) Task=`732fff57-52bc-4a0d-9e46-32a2ef53d559`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:24:07,374::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:24:07,375::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:24:07,375::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:24:07,375::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:24:07,376::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:24:07,376::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:24:07,376::task::993::Storage.TaskManager.Task::(_decref) Task=`732fff57-52bc-4a0d-9e46-32a2ef53d559`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:24:08,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:08,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132544 s, 77.3 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:24:09,856::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b9b35dc-01c8-4012-afcb-517de2165b9c`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:24:09,856::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:24:09,857::__init__::232::IOProcess::(_processLogs) (4953) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:24:09,859::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:09,859::__init__::232::IOProcess::(_processLogs) (4954) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:24:09,860::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:24:09,860::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b9b35dc-01c8-4012-afcb-517de2165b9c`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:24:09,860::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b9b35dc-01c8-4012-afcb-517de2165b9c`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:24:09,860::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:24:09,861::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:24:09,861::task::993::Storage.TaskManager.Task::(_decref) Task=`9b9b35dc-01c8-4012-afcb-517de2165b9c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:24:10,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:10,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133436 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:12,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:12,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132374 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:12,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:12,350::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:12,418::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:24:12,546::task::595::Storage.TaskManager.Task::(_updateState) Task=`73ef2b1d-c6ec-4523-8284-cfddd336dfcb`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:12,546::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:24:12,546::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418183', 'lastCheck': '6.5', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:12,547::task::1191::Storage.TaskManager.Task::(prepare) Task=`73ef2b1d-c6ec-4523-8284-cfddd336dfcb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418183', 'lastCheck': '6.5', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:12,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`73ef2b1d-c6ec-4523-8284-cfddd336dfcb`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:12,547::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:12,547::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:12,548::task::993::Storage.TaskManager.Task::(_decref) Task=`73ef2b1d-c6ec-4523-8284-cfddd336dfcb`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:24:12,566::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:14,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:14,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131852 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:16,097::__init__::232::IOProcess::(_processLogs) (4955) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:24:16,098::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:24:16,117::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424187 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:16,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:16,118::__init__::232::IOProcess::(_processLogs) (4956) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:16,120::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:16,120::__init__::232::IOProcess::(_processLogs) (4957) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:16,121::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:16,121::__init__::232::IOProcess::(_processLogs) (4958) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:16,122::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:16,122::__init__::232::IOProcess::(_processLogs) (4959) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:16,123::__init__::232::IOProcess::(_processLogs) (4959) Queuing response >Dummy-33::DEBUG::2014-08-11 14:24:16,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:16,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013236 s, 77.4 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:24:17,388::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:24:17,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b50d0bb-d3a2-4a51-abff-17c8a4c0c539`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:17,417::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:17,425::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:17,427::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:17,428::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:24:17,430::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:17,430::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b50d0bb-d3a2-4a51-abff-17c8a4c0c539`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:17,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b50d0bb-d3a2-4a51-abff-17c8a4c0c539`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:17,431::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:17,431::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:17,431::task::993::Storage.TaskManager.Task::(_decref) Task=`1b50d0bb-d3a2-4a51-abff-17c8a4c0c539`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:24:17,443::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:24:17,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`f688184f-1ea9-4047-8f08-50f61e97f0fb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:24:17,444::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:24:17,445::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3a644c5f-9888-4b50-aa2e-d81dbeff0217`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:24:17,445::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:24:17,445::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:24:17,446::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3a644c5f-9888-4b50-aa2e-d81dbeff0217`::Granted request >Thread-34::DEBUG::2014-08-11 14:24:17,446::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f688184f-1ea9-4047-8f08-50f61e97f0fb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:24:17,446::task::993::Storage.TaskManager.Task::(_decref) Task=`f688184f-1ea9-4047-8f08-50f61e97f0fb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:24:17,466::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:17,466::task::1191::Storage.TaskManager.Task::(prepare) Task=`f688184f-1ea9-4047-8f08-50f61e97f0fb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:17,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`f688184f-1ea9-4047-8f08-50f61e97f0fb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:24:17,467::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:24:17,467::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:24:17,467::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:24:17,468::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:24:17,468::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:24:17,468::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:24:17,468::task::993::Storage.TaskManager.Task::(_decref) Task=`f688184f-1ea9-4047-8f08-50f61e97f0fb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:24:18,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:18,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131271 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:20,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:20,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133205 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:22,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:22,436::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:22,437::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:22,438::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:22,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134244 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:24,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:24,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131211 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:26,124::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:26,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:26,126::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:24:26,127::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:24:26,127::__init__::232::IOProcess::(_processLogs) (4960) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:26,128::__init__::232::IOProcess::(_processLogs) (4960) Queuing response >Thread-16::DEBUG::2014-08-11 14:24:26,147::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419397 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:26,148::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:26,149::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:26,149::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:26,149::__init__::232::IOProcess::(_processLogs) (4961) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:26,150::__init__::232::IOProcess::(_processLogs) DEBUG|(4961) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:26,151::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:26,153::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:26,153::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:26,153::__init__::232::IOProcess::(_processLogs) (4962) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:26,154::__init__::232::IOProcess::(_processLogs) (4962) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:26,154::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:26,154::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:26,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:26,156::__init__::232::IOProcess::(_processLogs) (4963) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:26,156::__init__::232::IOProcess::(_processLogs) (4963) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:26,156::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:26,157::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:26,157::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:26,158::__init__::232::IOProcess::(_processLogs) (4964) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:26,158::__init__::232::IOProcess::(_processLogs) (4964) Queuing response >Dummy-33::DEBUG::2014-08-11 14:24:26,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:26,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132951 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:27,444::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:27,445::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:24:27,486::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:24:27,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a34a531-9ae9-49c5-8b66-9bfc952931ba`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:27,522::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:27,522::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:24:27,535::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:27,536::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a34a531-9ae9-49c5-8b66-9bfc952931ba`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:27,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a34a531-9ae9-49c5-8b66-9bfc952931ba`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:27,536::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:27,536::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:27,537::task::993::Storage.TaskManager.Task::(_decref) Task=`4a34a531-9ae9-49c5-8b66-9bfc952931ba`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:24:27,547::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:24:27,548::task::595::Storage.TaskManager.Task::(_updateState) Task=`aaab3baa-e011-47cb-858b-2eb23dad0296`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:24:27,548::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:24:27,549::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`436917b6-61af-41b8-92b8-c634507da127`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:24:27,549::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:24:27,549::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:24:27,550::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`436917b6-61af-41b8-92b8-c634507da127`::Granted request >Thread-34::DEBUG::2014-08-11 14:24:27,550::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aaab3baa-e011-47cb-858b-2eb23dad0296`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:24:27,550::task::993::Storage.TaskManager.Task::(_decref) Task=`aaab3baa-e011-47cb-858b-2eb23dad0296`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:24:27,570::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:27,571::task::1191::Storage.TaskManager.Task::(prepare) Task=`aaab3baa-e011-47cb-858b-2eb23dad0296`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:27,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`aaab3baa-e011-47cb-858b-2eb23dad0296`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:24:27,571::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:24:27,571::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:24:27,572::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:24:27,572::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:24:27,572::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:24:27,572::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:24:27,573::task::993::Storage.TaskManager.Task::(_decref) Task=`aaab3baa-e011-47cb-858b-2eb23dad0296`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:24:27,659::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4039839-d500-456b-bedb-e7b73794cf4e`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:27,659::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:24:27,659::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419397', 'lastCheck': '1.5', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:27,660::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4039839-d500-456b-bedb-e7b73794cf4e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419397', 'lastCheck': '1.5', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:27,660::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4039839-d500-456b-bedb-e7b73794cf4e`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:27,660::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:27,660::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:27,660::task::993::Storage.TaskManager.Task::(_decref) Task=`f4039839-d500-456b-bedb-e7b73794cf4e`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:24:27,680::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:28,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:28,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132946 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:30,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:30,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133229 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:32,531::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:32,538::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:32,623::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:32,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:32,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132115 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:34,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:34,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130996 s, 78.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:36,159::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:36,160::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:24:36,161::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:24:36,162::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:36,179::__init__::232::IOProcess::(_processLogs) (4965) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:36,179::__init__::232::IOProcess::(_processLogs) (4965) Queuing response >Thread-16::DEBUG::2014-08-11 14:24:36,181::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432222 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:36,182::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:36,182::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:36,182::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:36,183::__init__::232::IOProcess::(_processLogs) (4966) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:36,183::__init__::232::IOProcess::(_processLogs) DEBUG|(4966) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:36,185::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:36,185::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:36,186::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:36,186::__init__::232::IOProcess::(_processLogs) (4967) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:36,186::__init__::232::IOProcess::(_processLogs) (4967) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:36,187::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:36,188::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:36,188::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:36,188::__init__::232::IOProcess::(_processLogs) (4968) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:36,188::__init__::232::IOProcess::(_processLogs) (4968) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:36,189::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:36,189::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:36,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:36,190::__init__::232::IOProcess::(_processLogs) (4969) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:36,191::__init__::232::IOProcess::(_processLogs) (4969) Queuing response >Dummy-33::DEBUG::2014-08-11 14:24:36,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:37,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0149636 s, 68.4 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:24:37,586::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:24:37,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf2f58e8-d742-4bf9-b804-2448993cad50`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:37,617::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:24:37,630::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:37,631::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf2f58e8-d742-4bf9-b804-2448993cad50`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:37,631::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf2f58e8-d742-4bf9-b804-2448993cad50`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:37,631::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:37,631::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:37,632::task::993::Storage.TaskManager.Task::(_decref) Task=`cf2f58e8-d742-4bf9-b804-2448993cad50`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:37,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:37,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:37,636::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:24:37,644::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:24:37,644::task::595::Storage.TaskManager.Task::(_updateState) Task=`05fb6689-aa23-470a-b651-ea73f2b59480`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:24:37,644::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:24:37,645::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b48c1e64-bac8-4c67-b0a0-c5a3783bc4d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:24:37,645::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:24:37,646::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:24:37,646::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b48c1e64-bac8-4c67-b0a0-c5a3783bc4d4`::Granted request >Thread-34::DEBUG::2014-08-11 14:24:37,647::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`05fb6689-aa23-470a-b651-ea73f2b59480`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:24:37,647::task::993::Storage.TaskManager.Task::(_decref) Task=`05fb6689-aa23-470a-b651-ea73f2b59480`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:24:37,667::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:37,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`05fb6689-aa23-470a-b651-ea73f2b59480`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085326336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:37,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`05fb6689-aa23-470a-b651-ea73f2b59480`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:24:37,667::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:24:37,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:24:37,668::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:24:37,668::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:24:37,669::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:24:37,669::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:24:37,669::task::993::Storage.TaskManager.Task::(_decref) Task=`05fb6689-aa23-470a-b651-ea73f2b59480`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:24:39,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:39,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132159 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:41,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:41,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196603 s, 52.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:42,644::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:42,651::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:42,718::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:24:42,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfdb3130-350c-479f-825b-5c6683c8f430`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:42,776::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:24:42,776::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432222', 'lastCheck': '6.6', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:42,776::task::1191::Storage.TaskManager.Task::(prepare) Task=`cfdb3130-350c-479f-825b-5c6683c8f430`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432222', 'lastCheck': '6.6', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:42,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfdb3130-350c-479f-825b-5c6683c8f430`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:42,777::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:42,777::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:42,777::task::993::Storage.TaskManager.Task::(_decref) Task=`cfdb3130-350c-479f-825b-5c6683c8f430`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:24:42,798::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:43,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:43,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133227 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:45,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:45,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013131 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:46,195::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:46,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:46,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:46,196::__init__::232::IOProcess::(_processLogs) (4970) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:46,197::__init__::232::IOProcess::(_processLogs) DEBUG|(4970) Queuing response >Thread-16::DEBUG::2014-08-11 14:24:46,198::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:24:46,219::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00212185 s, 214 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:46,220::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:46,221::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:46,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:46,221::__init__::232::IOProcess::(_processLogs) (4971) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:46,222::__init__::232::IOProcess::(_processLogs) DEBUG|(4971) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:46,223::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:46,224::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:46,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:46,225::__init__::232::IOProcess::(_processLogs) (4972) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:46,225::__init__::232::IOProcess::(_processLogs) (4972) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:46,225::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:46,226::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:46,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:46,227::__init__::232::IOProcess::(_processLogs) (4973) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:46,227::__init__::232::IOProcess::(_processLogs) (4973) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:46,227::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:46,228::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:46,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:46,229::__init__::232::IOProcess::(_processLogs) (4974) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:46,230::__init__::232::IOProcess::(_processLogs) (4974) Queuing response >Dummy-33::DEBUG::2014-08-11 14:24:47,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:47,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131554 s, 77.8 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:24:47,681::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:24:47,682::task::595::Storage.TaskManager.Task::(_updateState) Task=`69a112df-b21b-4568-9d30-181737794d03`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:47,717::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:47,725::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:47,727::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:47,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:24:47,731::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:47,731::task::1191::Storage.TaskManager.Task::(prepare) Task=`69a112df-b21b-4568-9d30-181737794d03`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:47,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`69a112df-b21b-4568-9d30-181737794d03`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:47,732::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:47,732::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:47,732::task::993::Storage.TaskManager.Task::(_decref) Task=`69a112df-b21b-4568-9d30-181737794d03`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:24:47,743::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:24:47,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`e073c769-bad8-4849-8458-5731073e5bd2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:24:47,744::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:24:47,745::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5a5ec764-4727-4610-b344-c31afc4385e5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:24:47,745::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:24:47,746::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:24:47,746::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5a5ec764-4727-4610-b344-c31afc4385e5`::Granted request >Thread-34::DEBUG::2014-08-11 14:24:47,747::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e073c769-bad8-4849-8458-5731073e5bd2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:24:47,747::task::993::Storage.TaskManager.Task::(_decref) Task=`e073c769-bad8-4849-8458-5731073e5bd2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:24:47,766::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085260800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:47,767::task::1191::Storage.TaskManager.Task::(prepare) Task=`e073c769-bad8-4849-8458-5731073e5bd2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085260800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:47,767::task::595::Storage.TaskManager.Task::(_updateState) Task=`e073c769-bad8-4849-8458-5731073e5bd2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:24:47,767::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:24:47,768::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:24:47,768::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:24:47,768::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:24:47,768::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:24:47,769::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:24:47,769::task::993::Storage.TaskManager.Task::(_decref) Task=`e073c769-bad8-4849-8458-5731073e5bd2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:24:49,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:49,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134396 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:51,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:51,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134014 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:52,735::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:52,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:52,818::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:53,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:53,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132435 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:24:55,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:55,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132577 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:56,230::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:56,231::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:24:56,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:24:56,233::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:56,250::__init__::232::IOProcess::(_processLogs) (4975) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:56,251::__init__::232::IOProcess::(_processLogs) (4975) Queuing response >Thread-16::DEBUG::2014-08-11 14:24:56,252::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000545321 s, 834 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:24:56,253::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:56,253::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:56,254::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:56,254::__init__::232::IOProcess::(_processLogs) (4976) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:24:56,254::__init__::232::IOProcess::(_processLogs) DEBUG|(4976) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:56,256::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:56,257::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:56,257::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:56,257::__init__::232::IOProcess::(_processLogs) (4977) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:56,258::__init__::232::IOProcess::(_processLogs) (4977) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:56,258::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:56,259::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:56,259::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:56,259::__init__::232::IOProcess::(_processLogs) (4978) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:56,260::__init__::232::IOProcess::(_processLogs) (4978) Queuing response >Thread-14::DEBUG::2014-08-11 14:24:56,260::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:24:56,261::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:24:56,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:24:56,262::__init__::232::IOProcess::(_processLogs) (4979) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:24:56,262::__init__::232::IOProcess::(_processLogs) (4979) Queuing response >Dummy-33::DEBUG::2014-08-11 14:24:57,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:57,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131332 s, 78.0 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:24:57,781::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:24:57,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3aee829-c334-4280-8d47-a0c4330882bc`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:57,817::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:57,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:57,827::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:24:57,828::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:24:57,830::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:57,831::task::1191::Storage.TaskManager.Task::(prepare) Task=`c3aee829-c334-4280-8d47-a0c4330882bc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:24:57,831::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3aee829-c334-4280-8d47-a0c4330882bc`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:57,831::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:57,831::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:57,831::task::993::Storage.TaskManager.Task::(_decref) Task=`c3aee829-c334-4280-8d47-a0c4330882bc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:24:57,842::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:24:57,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`877a6c02-9860-408e-aa71-907fe24c72aa`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:24:57,843::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:24:57,844::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed82a70a-45e0-404c-8b79-884402b92f86`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:24:57,844::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:24:57,844::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:24:57,845::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed82a70a-45e0-404c-8b79-884402b92f86`::Granted request >Thread-34::DEBUG::2014-08-11 14:24:57,845::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`877a6c02-9860-408e-aa71-907fe24c72aa`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:24:57,845::task::993::Storage.TaskManager.Task::(_decref) Task=`877a6c02-9860-408e-aa71-907fe24c72aa`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:24:57,865::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085195264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:57,866::task::1191::Storage.TaskManager.Task::(prepare) Task=`877a6c02-9860-408e-aa71-907fe24c72aa`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085195264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:24:57,866::task::595::Storage.TaskManager.Task::(_updateState) Task=`877a6c02-9860-408e-aa71-907fe24c72aa`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:24:57,866::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:24:57,866::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:24:57,867::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:24:57,867::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:24:57,867::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:24:57,868::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:24:57,868::task::993::Storage.TaskManager.Task::(_decref) Task=`877a6c02-9860-408e-aa71-907fe24c72aa`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:24:57,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`0caf2fca-9e82-417b-9748-5a682abecc9e`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:24:57,889::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:24:57,890::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000545321', 'lastCheck': '1.6', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:57,890::task::1191::Storage.TaskManager.Task::(prepare) Task=`0caf2fca-9e82-417b-9748-5a682abecc9e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000545321', 'lastCheck': '1.6', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:24:57,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`0caf2fca-9e82-417b-9748-5a682abecc9e`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:24:57,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:24:57,891::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:24:57,891::task::993::Storage.TaskManager.Task::(_decref) Task=`0caf2fca-9e82-417b-9748-5a682abecc9e`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:24:57,910::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:24:59,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:24:59,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132652 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:01,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:01,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132902 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:02,837::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:02,842::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:02,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:25:03,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:03,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132776 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:05,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:05,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013215 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:06,263::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:06,263::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:06,264::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:06,264::__init__::232::IOProcess::(_processLogs) (4980) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:06,265::__init__::232::IOProcess::(_processLogs) DEBUG|(4980) Queuing response >Thread-16::DEBUG::2014-08-11 14:25:06,266::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:25:06,285::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414211 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:06,286::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:06,287::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:06,288::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:06,288::__init__::232::IOProcess::(_processLogs) (4981) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:06,288::__init__::232::IOProcess::(_processLogs) (4981) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:06,289::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:06,289::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:06,289::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:06,290::__init__::232::IOProcess::(_processLogs) (4982) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:06,290::__init__::232::IOProcess::(_processLogs) DEBUG|(4982) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:06,291::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:06,292::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:06,292::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:06,293::__init__::232::IOProcess::(_processLogs) (4983) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:06,293::__init__::232::IOProcess::(_processLogs) (4983) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:06,293::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:06,293::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:06,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:06,294::__init__::232::IOProcess::(_processLogs) (4984) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:06,294::__init__::232::IOProcess::(_processLogs) DEBUG|(4984) Queuing response >Dummy-33::DEBUG::2014-08-11 14:25:07,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:07,867::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:07,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:07,870::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:25:07,881::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:25:07,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`850e670b-77f0-489a-8d29-45e72f8d543e`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:25:07,882::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 14:25:07,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155305 s, 65.9 MB/s\n'; <rc> = 0 >Thread-89::INFO::2014-08-11 14:25:07,901::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:25:07,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`850e670b-77f0-489a-8d29-45e72f8d543e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:25:07,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`850e670b-77f0-489a-8d29-45e72f8d543e`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:25:07,904::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:25:07,905::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:25:07,906::task::993::Storage.TaskManager.Task::(_decref) Task=`850e670b-77f0-489a-8d29-45e72f8d543e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:25:07,919::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:25:07,920::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf320b86-4efa-44ba-b553-c87c7e1f4dca`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:25:07,920::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:25:07,921::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`97cccbca-0041-40d5-b7fd-8c462fa66675`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:25:07,921::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:25:07,921::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:25:07,922::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`97cccbca-0041-40d5-b7fd-8c462fa66675`::Granted request >Thread-34::DEBUG::2014-08-11 14:25:07,924::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bf320b86-4efa-44ba-b553-c87c7e1f4dca`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:25:07,925::task::993::Storage.TaskManager.Task::(_decref) Task=`bf320b86-4efa-44ba-b553-c87c7e1f4dca`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:25:07,944::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085195264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:07,945::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf320b86-4efa-44ba-b553-c87c7e1f4dca`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085195264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:07,945::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf320b86-4efa-44ba-b553-c87c7e1f4dca`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:25:07,945::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:25:07,945::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:25:07,946::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:25:07,946::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:25:07,947::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:25:07,947::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:25:07,947::task::993::Storage.TaskManager.Task::(_decref) Task=`bf320b86-4efa-44ba-b553-c87c7e1f4dca`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:25:09,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:09,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131668 s, 77.8 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:25:09,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`669a784c-7add-4fb8-ad92-176bd79cf690`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:25:09,968::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:25:09,971::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:09,972::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:09,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:09,974::__init__::232::IOProcess::(_processLogs) (4985) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:25:09,976::__init__::232::IOProcess::(_processLogs) DEBUG|(4985) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:09,978::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:09,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:09,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:09,981::__init__::232::IOProcess::(_processLogs) (4986) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:25:09,983::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:25:09,983::__init__::232::IOProcess::(_processLogs) DEBUG|(4986) Queuing response >Thread-55::DEBUG::2014-08-11 14:25:09,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`669a784c-7add-4fb8-ad92-176bd79cf690`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:25:09,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`669a784c-7add-4fb8-ad92-176bd79cf690`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:25:09,986::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:25:09,986::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:25:09,988::task::993::Storage.TaskManager.Task::(_decref) Task=`669a784c-7add-4fb8-ad92-176bd79cf690`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:25:12,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:12,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01324 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:12,876::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:12,878::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:12,918::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:25:13,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`43fd7e8b-c621-465a-8689-5f2fb935460d`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:25:13,005::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:25:13,006::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414211', 'lastCheck': '6.7', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:25:13,006::task::1191::Storage.TaskManager.Task::(prepare) Task=`43fd7e8b-c621-465a-8689-5f2fb935460d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414211', 'lastCheck': '6.7', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:25:13,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`43fd7e8b-c621-465a-8689-5f2fb935460d`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:25:13,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:25:13,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:25:13,007::task::993::Storage.TaskManager.Task::(_decref) Task=`43fd7e8b-c621-465a-8689-5f2fb935460d`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:25:13,026::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:25:14,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:14,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132168 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:16,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:16,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013267 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:16,296::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:16,297::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:16,297::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:16,297::__init__::232::IOProcess::(_processLogs) (4987) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:16,298::__init__::232::IOProcess::(_processLogs) DEBUG|(4987) Queuing response >Thread-16::DEBUG::2014-08-11 14:25:16,299::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:25:16,318::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418805 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:16,319::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:16,319::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:16,320::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:16,320::__init__::232::IOProcess::(_processLogs) (4988) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:16,321::__init__::232::IOProcess::(_processLogs) DEBUG|(4988) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:16,322::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:16,322::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:16,323::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:16,323::__init__::232::IOProcess::(_processLogs) (4989) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:16,324::__init__::232::IOProcess::(_processLogs) (4989) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:16,324::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:16,324::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:16,324::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:16,325::__init__::232::IOProcess::(_processLogs) (4990) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:16,325::__init__::232::IOProcess::(_processLogs) DEBUG|(4990) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:16,326::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:16,327::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:16,327::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:16,328::__init__::232::IOProcess::(_processLogs) (4991) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:16,328::__init__::232::IOProcess::(_processLogs) (4991) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:17,921::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:17,923::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:25:17,960::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:25:18,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5962388-0f70-4a7d-b16d-90b05bd52771`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:25:18,022::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:18,023::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:25:18,036::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:25:18,037::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5962388-0f70-4a7d-b16d-90b05bd52771`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:25:18,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5962388-0f70-4a7d-b16d-90b05bd52771`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:25:18,037::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:25:18,037::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:25:18,038::task::993::Storage.TaskManager.Task::(_decref) Task=`b5962388-0f70-4a7d-b16d-90b05bd52771`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:25:18,049::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:25:18,050::task::595::Storage.TaskManager.Task::(_updateState) Task=`49146ac8-9287-4718-b967-5d9313a58a37`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:25:18,050::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:25:18,051::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cb044dba-f2ff-4a7e-805d-879daf18ff4c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:25:18,051::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:25:18,052::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:25:18,052::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cb044dba-f2ff-4a7e-805d-879daf18ff4c`::Granted request >Thread-34::DEBUG::2014-08-11 14:25:18,052::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`49146ac8-9287-4718-b967-5d9313a58a37`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:25:18,053::task::993::Storage.TaskManager.Task::(_decref) Task=`49146ac8-9287-4718-b967-5d9313a58a37`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:25:18,072::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085195264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:18,073::task::1191::Storage.TaskManager.Task::(prepare) Task=`49146ac8-9287-4718-b967-5d9313a58a37`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085195264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:18,073::task::595::Storage.TaskManager.Task::(_updateState) Task=`49146ac8-9287-4718-b967-5d9313a58a37`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:25:18,073::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:25:18,073::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:25:18,074::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:25:18,074::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:25:18,074::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:25:18,074::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:25:18,075::task::993::Storage.TaskManager.Task::(_decref) Task=`49146ac8-9287-4718-b967-5d9313a58a37`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:25:18,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:18,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0290061 s, 35.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:20,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:20,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132647 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:22,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:22,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131668 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:23,031::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:23,032::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:23,119::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:25:24,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:24,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132671 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:26,328::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:26,329::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:26,329::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:26,329::__init__::232::IOProcess::(_processLogs) (4992) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:26,330::__init__::232::IOProcess::(_processLogs) (4992) Queuing response >Thread-16::DEBUG::2014-08-11 14:25:26,330::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:25:26,349::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420395 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:26,350::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:26,350::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:26,351::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:26,351::__init__::232::IOProcess::(_processLogs) (4993) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:26,352::__init__::232::IOProcess::(_processLogs) DEBUG|(4993) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:26,353::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:26,354::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:26,354::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:26,355::__init__::232::IOProcess::(_processLogs) (4994) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:26,355::__init__::232::IOProcess::(_processLogs) (4994) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:26,355::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:26,356::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:26,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:26,357::__init__::232::IOProcess::(_processLogs) (4995) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:26,357::__init__::232::IOProcess::(_processLogs) (4995) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:26,357::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:26,358::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:26,358::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:26,359::__init__::232::IOProcess::(_processLogs) (4996) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:26,359::__init__::232::IOProcess::(_processLogs) (4996) Queuing response >Dummy-33::DEBUG::2014-08-11 14:25:26,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:26,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132917 s, 77.0 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:25:28,086::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:25:28,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a3db5c8-ef8d-45c8-8bfd-a3746ff98fb1`::moving from state init -> state preparing >Detector thread::DEBUG::2014-08-11 14:25:28,115::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:60656 >Thread-89::INFO::2014-08-11 14:25:28,120::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Detector thread::DEBUG::2014-08-11 14:25:28,121::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:60656 >Detector thread::DEBUG::2014-08-11 14:25:28,122::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:60656 >Detector thread::DEBUG::2014-08-11 14:25:28,123::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 60656) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:28,127::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:28,131::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:28,135::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::DEBUG::2014-08-11 14:25:28,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`5122ab7d-9f6e-4aed-a99a-d01e42a35406`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:25:28,137::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::INFO::2014-08-11 14:25:28,138::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::DEBUG::2014-08-11 14:25:28,139::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a3db5c8-ef8d-45c8-8bfd-a3746ff98fb1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:25:28,140::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a3db5c8-ef8d-45c8-8bfd-a3746ff98fb1`::moving from state preparing -> state finished >Thread-90::INFO::2014-08-11 14:25:28,140::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420395', 'lastCheck': '1.8', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:25:28,141::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:25:28,142::task::1191::Storage.TaskManager.Task::(prepare) Task=`5122ab7d-9f6e-4aed-a99a-d01e42a35406`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420395', 'lastCheck': '1.8', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:25:28,142::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:25:28,143::task::993::Storage.TaskManager.Task::(_decref) Task=`2a3db5c8-ef8d-45c8-8bfd-a3746ff98fb1`::ref 0 aborting False >Thread-90::DEBUG::2014-08-11 14:25:28,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`5122ab7d-9f6e-4aed-a99a-d01e42a35406`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:25:28,144::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:25:28,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:25:28,146::task::993::Storage.TaskManager.Task::(_decref) Task=`5122ab7d-9f6e-4aed-a99a-d01e42a35406`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:25:28,158::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:25:28,159::task::595::Storage.TaskManager.Task::(_updateState) Task=`e37c1ce8-5896-40d1-a8bc-84464ac190b3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:25:28,159::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:25:28,160::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`af7ae40d-b8aa-41ea-ad94-2759b83243eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:25:28,160::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:25:28,161::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:25:28,161::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`af7ae40d-b8aa-41ea-ad94-2759b83243eb`::Granted request >Thread-34::DEBUG::2014-08-11 14:25:28,162::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e37c1ce8-5896-40d1-a8bc-84464ac190b3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:25:28,162::task::993::Storage.TaskManager.Task::(_decref) Task=`e37c1ce8-5896-40d1-a8bc-84464ac190b3`::ref 1 aborting False >Thread-90::DEBUG::2014-08-11 14:25:28,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:25:28,182::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:28,182::task::1191::Storage.TaskManager.Task::(prepare) Task=`e37c1ce8-5896-40d1-a8bc-84464ac190b3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:28,182::task::595::Storage.TaskManager.Task::(_updateState) Task=`e37c1ce8-5896-40d1-a8bc-84464ac190b3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:25:28,183::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:25:28,183::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:25:28,183::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:25:28,184::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:25:28,184::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:25:28,184::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:25:28,184::task::993::Storage.TaskManager.Task::(_decref) Task=`e37c1ce8-5896-40d1-a8bc-84464ac190b3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:25:28,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:28,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132668 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:30,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:30,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132671 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:32,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:32,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269565 s, 38.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:33,143::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:33,144::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:33,221::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:25:34,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:34,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133185 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:36,360::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:36,360::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:36,361::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:36,361::__init__::232::IOProcess::(_processLogs) (4997) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:36,361::__init__::232::IOProcess::(_processLogs) (4997) Queuing response >Thread-16::DEBUG::2014-08-11 14:25:36,362::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:25:36,380::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419442 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:36,382::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:36,382::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:36,382::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:36,383::__init__::232::IOProcess::(_processLogs) (4998) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:36,383::__init__::232::IOProcess::(_processLogs) DEBUG|(4998) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:36,385::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:36,385::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:36,386::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:36,386::__init__::232::IOProcess::(_processLogs) (4999) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:36,386::__init__::232::IOProcess::(_processLogs) (4999) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:36,387::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:36,387::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:36,388::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:36,388::__init__::232::IOProcess::(_processLogs) (5000) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:36,389::__init__::232::IOProcess::(_processLogs) (5000) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:36,389::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:36,390::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:36,390::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:36,391::__init__::232::IOProcess::(_processLogs) (5001) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:36,391::__init__::232::IOProcess::(_processLogs) (5001) Queuing response >Dummy-33::DEBUG::2014-08-11 14:25:37,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:37,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133128 s, 76.9 MB/s\n'; <rc> = 0 >Thread-90::DEBUG::2014-08-11 14:25:38,196::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-90::DEBUG::2014-08-11 14:25:38,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f2e2e22-0496-458d-961e-4a33d3208424`::moving from state init -> state preparing >Thread-90::INFO::2014-08-11 14:25:38,220::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:38,227::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:38,229::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:38,230::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::INFO::2014-08-11 14:25:38,233::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:25:38,234::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f2e2e22-0496-458d-961e-4a33d3208424`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:25:38,234::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f2e2e22-0496-458d-961e-4a33d3208424`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:25:38,234::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:25:38,234::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:25:38,235::task::993::Storage.TaskManager.Task::(_decref) Task=`5f2e2e22-0496-458d-961e-4a33d3208424`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:25:38,245::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:25:38,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cbf6548-0221-4680-8b55-4404cd354c69`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:25:38,246::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:25:38,247::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ea3db536-8450-4661-8399-1898d1a29985`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:25:38,247::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:25:38,248::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:25:38,248::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ea3db536-8450-4661-8399-1898d1a29985`::Granted request >Thread-34::DEBUG::2014-08-11 14:25:38,248::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4cbf6548-0221-4680-8b55-4404cd354c69`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:25:38,249::task::993::Storage.TaskManager.Task::(_decref) Task=`4cbf6548-0221-4680-8b55-4404cd354c69`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:25:38,268::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:38,269::task::1191::Storage.TaskManager.Task::(prepare) Task=`4cbf6548-0221-4680-8b55-4404cd354c69`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:38,269::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cbf6548-0221-4680-8b55-4404cd354c69`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:25:38,269::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:25:38,269::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:25:38,270::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:25:38,270::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:25:38,270::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:25:38,271::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:25:38,271::task::993::Storage.TaskManager.Task::(_decref) Task=`4cbf6548-0221-4680-8b55-4404cd354c69`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:25:39,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:39,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131862 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:41,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:41,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132579 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:43,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:43,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132218 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:43,234::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:43,235::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:43,236::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::DEBUG::2014-08-11 14:25:43,269::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8919e5b-4850-4e53-aed5-def4f4208e4c`::moving from state init -> state preparing >Thread-90::INFO::2014-08-11 14:25:43,269::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-90::INFO::2014-08-11 14:25:43,269::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419442', 'lastCheck': '6.9', 'valid': True}} >Thread-90::DEBUG::2014-08-11 14:25:43,270::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8919e5b-4850-4e53-aed5-def4f4208e4c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419442', 'lastCheck': '6.9', 'valid': True}} >Thread-90::DEBUG::2014-08-11 14:25:43,270::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8919e5b-4850-4e53-aed5-def4f4208e4c`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:25:43,270::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:25:43,270::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:25:43,270::task::993::Storage.TaskManager.Task::(_decref) Task=`e8919e5b-4850-4e53-aed5-def4f4208e4c`::ref 0 aborting False >Thread-90::DEBUG::2014-08-11 14:25:43,289::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:25:45,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:45,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133553 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:46,392::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:46,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:25:46,394::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:25:46,394::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:46,412::__init__::232::IOProcess::(_processLogs) (5002) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:46,412::__init__::232::IOProcess::(_processLogs) (5002) Queuing response >Thread-16::DEBUG::2014-08-11 14:25:46,413::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000409591 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:46,414::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:46,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:46,415::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:46,415::__init__::232::IOProcess::(_processLogs) (5003) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:46,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5003) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:46,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:46,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:46,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:46,418::__init__::232::IOProcess::(_processLogs) (5004) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:46,418::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5004) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:46,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:46,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:46,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:46,419::__init__::232::IOProcess::(_processLogs) (5005) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:46,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5005) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:46,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:46,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:46,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:46,422::__init__::232::IOProcess::(_processLogs) (5006) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:46,423::__init__::232::IOProcess::(_processLogs) (5006) Queuing response >Dummy-33::DEBUG::2014-08-11 14:25:47,282::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:47,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131832 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:48,243::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:48,244::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::DEBUG::2014-08-11 14:25:48,284::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-90::DEBUG::2014-08-11 14:25:48,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9126fed-adcd-4718-ab01-b49e87c59a83`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:48,327::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::INFO::2014-08-11 14:25:48,327::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-90::INFO::2014-08-11 14:25:48,342::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:25:48,342::task::1191::Storage.TaskManager.Task::(prepare) Task=`f9126fed-adcd-4718-ab01-b49e87c59a83`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:25:48,342::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9126fed-adcd-4718-ab01-b49e87c59a83`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:25:48,343::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:25:48,343::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:25:48,343::task::993::Storage.TaskManager.Task::(_decref) Task=`f9126fed-adcd-4718-ab01-b49e87c59a83`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:25:48,354::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:25:48,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`45537663-0a76-4eed-bebe-3400486838ee`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:25:48,355::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:25:48,356::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`24426c33-fd70-4ef0-84ec-2bc75c6caf72`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:25:48,356::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:25:48,357::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:25:48,357::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`24426c33-fd70-4ef0-84ec-2bc75c6caf72`::Granted request >Thread-34::DEBUG::2014-08-11 14:25:48,357::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`45537663-0a76-4eed-bebe-3400486838ee`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:25:48,358::task::993::Storage.TaskManager.Task::(_decref) Task=`45537663-0a76-4eed-bebe-3400486838ee`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:25:48,377::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:48,378::task::1191::Storage.TaskManager.Task::(prepare) Task=`45537663-0a76-4eed-bebe-3400486838ee`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:48,378::task::595::Storage.TaskManager.Task::(_updateState) Task=`45537663-0a76-4eed-bebe-3400486838ee`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:25:48,378::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:25:48,378::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:25:48,379::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:25:48,379::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:25:48,379::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:25:48,379::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:25:48,380::task::993::Storage.TaskManager.Task::(_decref) Task=`45537663-0a76-4eed-bebe-3400486838ee`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:25:49,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:49,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133089 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:51,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:51,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132334 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:53,331::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:53,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:53,424::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:25:53,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:53,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131972 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:25:55,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:55,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131673 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:56,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:56,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:56,425::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:56,425::__init__::232::IOProcess::(_processLogs) (5007) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:56,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5007) Queuing response >Thread-16::DEBUG::2014-08-11 14:25:56,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:25:56,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420519 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:25:56,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:56,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:56,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:56,448::__init__::232::IOProcess::(_processLogs) (5008) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:25:56,448::__init__::232::IOProcess::(_processLogs) (5008) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:56,449::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:56,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:56,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:56,451::__init__::232::IOProcess::(_processLogs) (5009) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:56,451::__init__::232::IOProcess::(_processLogs) (5009) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:56,451::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:56,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:56,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:56,453::__init__::232::IOProcess::(_processLogs) (5010) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:56,453::__init__::232::IOProcess::(_processLogs) (5010) Queuing response >Thread-14::DEBUG::2014-08-11 14:25:56,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:25:56,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:25:56,454::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:25:56,456::__init__::232::IOProcess::(_processLogs) (5011) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:25:56,456::__init__::232::IOProcess::(_processLogs) (5011) Queuing response >Dummy-33::DEBUG::2014-08-11 14:25:57,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:57,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179813 s, 56.9 MB/s\n'; <rc> = 0 >Thread-90::DEBUG::2014-08-11 14:25:58,397::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-90::DEBUG::2014-08-11 14:25:58,398::task::595::Storage.TaskManager.Task::(_updateState) Task=`53f9cb5c-dc9a-4877-bb49-58d80a476f51`::moving from state init -> state preparing >Thread-89::DEBUG::2014-08-11 14:25:58,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`1034ec99-61bf-46cc-95a6-2d6b990e2ada`::moving from state init -> state preparing >Thread-90::INFO::2014-08-11 14:25:58,424::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:25:58,425::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:25:58,426::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420519', 'lastCheck': '2.0', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:25:58,426::task::1191::Storage.TaskManager.Task::(prepare) Task=`1034ec99-61bf-46cc-95a6-2d6b990e2ada`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420519', 'lastCheck': '2.0', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:25:58,427::task::595::Storage.TaskManager.Task::(_updateState) Task=`1034ec99-61bf-46cc-95a6-2d6b990e2ada`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:25:58,427::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:25:58,427::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:25:58,427::task::993::Storage.TaskManager.Task::(_decref) Task=`1034ec99-61bf-46cc-95a6-2d6b990e2ada`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:58,432::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:58,436::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:25:58,439::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::INFO::2014-08-11 14:25:58,444::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:25:58,445::task::1191::Storage.TaskManager.Task::(prepare) Task=`53f9cb5c-dc9a-4877-bb49-58d80a476f51`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:25:58,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`53f9cb5c-dc9a-4877-bb49-58d80a476f51`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:25:58,445::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:25:58,446::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:25:58,446::task::993::Storage.TaskManager.Task::(_decref) Task=`53f9cb5c-dc9a-4877-bb49-58d80a476f51`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:25:58,460::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:25:58,461::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6b5e34d-b6aa-41f8-94b8-9250bc6d816b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:25:58,461::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:25:58,462::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`67b7c78b-fcb8-4c67-b58c-dc9e81041fb0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:25:58,462::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:25:58,463::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:25:58,463::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`67b7c78b-fcb8-4c67-b58c-dc9e81041fb0`::Granted request >Thread-34::DEBUG::2014-08-11 14:25:58,463::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6b5e34d-b6aa-41f8-94b8-9250bc6d816b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:25:58,464::task::993::Storage.TaskManager.Task::(_decref) Task=`b6b5e34d-b6aa-41f8-94b8-9250bc6d816b`::ref 1 aborting False >Thread-90::DEBUG::2014-08-11 14:25:58,476::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:25:58,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:58,484::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6b5e34d-b6aa-41f8-94b8-9250bc6d816b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:25:58,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6b5e34d-b6aa-41f8-94b8-9250bc6d816b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:25:58,484::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:25:58,485::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:25:58,485::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:25:58,485::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:25:58,485::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:25:58,486::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:25:58,486::task::993::Storage.TaskManager.Task::(_decref) Task=`b6b5e34d-b6aa-41f8-94b8-9250bc6d816b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:25:59,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:25:59,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132007 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:01,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:01,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132997 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:03,448::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:03,449::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:03,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:26:03,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:03,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131885 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:05,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:05,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132061 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:06,456::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:06,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:06,457::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:06,457::__init__::232::IOProcess::(_processLogs) (5012) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:06,458::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5012) Queuing response >Thread-16::DEBUG::2014-08-11 14:26:06,459::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:26:06,479::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417568 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:06,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:06,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:06,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:06,481::__init__::232::IOProcess::(_processLogs) (5013) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:06,481::__init__::232::IOProcess::(_processLogs) (5013) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:06,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:06,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:06,483::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:06,483::__init__::232::IOProcess::(_processLogs) (5014) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:06,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5014) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:06,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:06,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:06,486::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:06,486::__init__::232::IOProcess::(_processLogs) (5015) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:06,486::__init__::232::IOProcess::(_processLogs) (5015) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:06,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:06,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:06,487::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:06,487::__init__::232::IOProcess::(_processLogs) (5016) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:06,488::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5016) Queuing response >Dummy-33::DEBUG::2014-08-11 14:26:07,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:07,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132278 s, 77.4 MB/s\n'; <rc> = 0 >Thread-90::DEBUG::2014-08-11 14:26:08,500::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-90::DEBUG::2014-08-11 14:26:08,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`897ffd41-60fd-455d-a495-9b5a6ca11234`::moving from state init -> state preparing >Thread-90::INFO::2014-08-11 14:26:08,525::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:08,532::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:08,534::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:08,535::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::INFO::2014-08-11 14:26:08,537::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:26:08,538::task::1191::Storage.TaskManager.Task::(prepare) Task=`897ffd41-60fd-455d-a495-9b5a6ca11234`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:26:08,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`897ffd41-60fd-455d-a495-9b5a6ca11234`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:26:08,538::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:26:08,538::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:26:08,539::task::993::Storage.TaskManager.Task::(_decref) Task=`897ffd41-60fd-455d-a495-9b5a6ca11234`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:26:08,549::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:26:08,550::task::595::Storage.TaskManager.Task::(_updateState) Task=`65bba5c8-450b-42d7-9742-50b7d2c6117e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:26:08,550::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:26:08,551::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`474edceb-97d2-4fcf-bf65-f0e0a256d33e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:26:08,551::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:26:08,551::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:26:08,552::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`474edceb-97d2-4fcf-bf65-f0e0a256d33e`::Granted request >Thread-34::DEBUG::2014-08-11 14:26:08,552::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`65bba5c8-450b-42d7-9742-50b7d2c6117e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:26:08,552::task::993::Storage.TaskManager.Task::(_decref) Task=`65bba5c8-450b-42d7-9742-50b7d2c6117e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:26:08,572::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:08,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`65bba5c8-450b-42d7-9742-50b7d2c6117e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:08,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`65bba5c8-450b-42d7-9742-50b7d2c6117e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:26:08,573::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:26:08,573::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:26:08,573::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:26:08,574::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:26:08,574::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:26:08,574::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:26:08,575::task::993::Storage.TaskManager.Task::(_decref) Task=`65bba5c8-450b-42d7-9742-50b7d2c6117e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:26:09,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:09,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134531 s, 76.1 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:26:10,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`cefb8d4c-9092-4449-83e9-4a739c7adb14`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:26:10,113::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:26:10,114::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:10,114::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:10,114::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:10,115::__init__::232::IOProcess::(_processLogs) (5017) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:26:10,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5017) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:10,116::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:10,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:26:10,117::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:26:10,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:26:10,119::task::1191::Storage.TaskManager.Task::(prepare) Task=`cefb8d4c-9092-4449-83e9-4a739c7adb14`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:26:10,120::__init__::232::IOProcess::(_processLogs) (5018) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:26:10,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`cefb8d4c-9092-4449-83e9-4a739c7adb14`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:26:10,121::__init__::232::IOProcess::(_processLogs) (5018) Queuing response >Thread-55::DEBUG::2014-08-11 14:26:10,121::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:26:10,122::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:26:10,123::task::993::Storage.TaskManager.Task::(_decref) Task=`cefb8d4c-9092-4449-83e9-4a739c7adb14`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:26:11,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:11,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132219 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:13,542::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:13,544::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:13,545::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::DEBUG::2014-08-11 14:26:13,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6be62b3-d38f-4c2d-b2df-6264e8e41ce5`::moving from state init -> state preparing >Thread-90::INFO::2014-08-11 14:26:13,574::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-90::INFO::2014-08-11 14:26:13,574::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417568', 'lastCheck': '7.1', 'valid': True}} >Thread-90::DEBUG::2014-08-11 14:26:13,575::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6be62b3-d38f-4c2d-b2df-6264e8e41ce5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417568', 'lastCheck': '7.1', 'valid': True}} >Thread-90::DEBUG::2014-08-11 14:26:13,575::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6be62b3-d38f-4c2d-b2df-6264e8e41ce5`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:26:13,575::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:26:13,575::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:26:13,576::task::993::Storage.TaskManager.Task::(_decref) Task=`f6be62b3-d38f-4c2d-b2df-6264e8e41ce5`::ref 0 aborting False >Thread-90::DEBUG::2014-08-11 14:26:13,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:26:13,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:14,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131614 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:16,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:16,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013259 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:16,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:16,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:16,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:16,490::__init__::232::IOProcess::(_processLogs) (5019) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:26:16,491::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:26:16,510::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421348 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:16,511::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:16,511::__init__::232::IOProcess::(_processLogs) (5020) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:16,512::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:16,513::__init__::232::IOProcess::(_processLogs) (5021) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:16,513::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5021) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:16,514::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:16,514::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:16,515::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:16,515::__init__::232::IOProcess::(_processLogs) (5022) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:16,515::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5022) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:16,516::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:16,517::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:16,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:16,518::__init__::232::IOProcess::(_processLogs) (5023) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:16,518::__init__::232::IOProcess::(_processLogs) (5023) Queuing response >Dummy-33::DEBUG::2014-08-11 14:26:18,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:18,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013221 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:18,552::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:18,553::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::DEBUG::2014-08-11 14:26:18,587::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-90::DEBUG::2014-08-11 14:26:18,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`139a25e3-5c87-4cf1-9020-0cc8e9298c2d`::moving from state init -> state preparing >Thread-90::INFO::2014-08-11 14:26:18,629::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:18,629::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::INFO::2014-08-11 14:26:18,643::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:26:18,643::task::1191::Storage.TaskManager.Task::(prepare) Task=`139a25e3-5c87-4cf1-9020-0cc8e9298c2d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-90::DEBUG::2014-08-11 14:26:18,643::task::595::Storage.TaskManager.Task::(_updateState) Task=`139a25e3-5c87-4cf1-9020-0cc8e9298c2d`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:26:18,643::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-08-11 14:26:18,644::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-08-11 14:26:18,644::task::993::Storage.TaskManager.Task::(_decref) Task=`139a25e3-5c87-4cf1-9020-0cc8e9298c2d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:26:18,654::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:26:18,655::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2f6ca82-16f7-4269-acb7-e037ab224a40`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:26:18,656::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:26:18,657::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`345bc2a1-63f3-4647-8ab2-0e20f1c7a347`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:26:18,657::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:26:18,657::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:26:18,657::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`345bc2a1-63f3-4647-8ab2-0e20f1c7a347`::Granted request >Thread-34::DEBUG::2014-08-11 14:26:18,659::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c2f6ca82-16f7-4269-acb7-e037ab224a40`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:26:18,660::task::993::Storage.TaskManager.Task::(_decref) Task=`c2f6ca82-16f7-4269-acb7-e037ab224a40`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:26:18,680::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:18,681::task::1191::Storage.TaskManager.Task::(prepare) Task=`c2f6ca82-16f7-4269-acb7-e037ab224a40`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:18,681::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2f6ca82-16f7-4269-acb7-e037ab224a40`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:26:18,681::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:26:18,681::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:26:18,682::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:26:18,682::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:26:18,682::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:26:18,683::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:26:18,683::task::993::Storage.TaskManager.Task::(_decref) Task=`c2f6ca82-16f7-4269-acb7-e037ab224a40`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:26:20,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:20,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133675 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:22,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:22,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131482 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:23,632::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:23,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:23,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:26:24,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:24,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132846 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:26,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:26,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172386 s, 59.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:26,519::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:26,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:26,520::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:26,520::__init__::232::IOProcess::(_processLogs) (5024) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:26:26,521::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:26:26,541::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041426 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:26,542::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:26,542::__init__::232::IOProcess::(_processLogs) (5025) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:26,544::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:26,544::__init__::232::IOProcess::(_processLogs) (5026) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:26,544::__init__::232::IOProcess::(_processLogs) DEBUG|(5026) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:26,545::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:26,546::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:26,547::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:26,547::__init__::232::IOProcess::(_processLogs) (5027) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:26,547::__init__::232::IOProcess::(_processLogs) (5027) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:26,547::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:26,548::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:26,548::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:26,549::__init__::232::IOProcess::(_processLogs) (5028) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:26,550::__init__::232::IOProcess::(_processLogs) (5028) Queuing response >Dummy-33::DEBUG::2014-08-11 14:26:28,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:28,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01326 s, 77.2 MB/s\n'; <rc> = 0 >Thread-90::DEBUG::2014-08-11 14:26:28,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`a9cf66ca-5bf0-44ed-9564-7c366f5ebe05`::moving from state init -> state preparing >Thread-90::INFO::2014-08-11 14:26:28,692::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::DEBUG::2014-08-11 14:26:28,696::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-90::INFO::2014-08-11 14:26:28,727::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041426', 'lastCheck': '2.2', 'valid': True}} >Thread-90::DEBUG::2014-08-11 14:26:28,729::task::1191::Storage.TaskManager.Task::(prepare) Task=`a9cf66ca-5bf0-44ed-9564-7c366f5ebe05`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041426', 'lastCheck': '2.2', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:26:28,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc7b60d2-ff1d-4f75-b8bc-8dca8f2fbdfc`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:28,731::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::DEBUG::2014-08-11 14:26:28,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`a9cf66ca-5bf0-44ed-9564-7c366f5ebe05`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-08-11 14:26:28,732::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::INFO::2014-08-11 14:26:28,733::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-90::DEBUG::2014-08-11 14:26:28,734::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:28,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-90::DEBUG::2014-08-11 14:26:28,736::task::993::Storage.TaskManager.Task::(_decref) Task=`a9cf66ca-5bf0-44ed-9564-7c366f5ebe05`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:28,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:26:28,752::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:26:28,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc7b60d2-ff1d-4f75-b8bc-8dca8f2fbdfc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:26:28,753::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc7b60d2-ff1d-4f75-b8bc-8dca8f2fbdfc`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:26:28,753::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:26:28,753::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:26:28,753::task::993::Storage.TaskManager.Task::(_decref) Task=`fc7b60d2-ff1d-4f75-b8bc-8dca8f2fbdfc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:26:28,768::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:26:28,768::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dda8ce0-70ae-43da-9fca-35325b9baf21`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:26:28,769::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:26:28,769::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7189b726-5885-4a45-b169-5b7653dbb1be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:26:28,770::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:26:28,770::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:26:28,770::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7189b726-5885-4a45-b169-5b7653dbb1be`::Granted request >Thread-34::DEBUG::2014-08-11 14:26:28,771::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9dda8ce0-70ae-43da-9fca-35325b9baf21`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:26:28,771::task::993::Storage.TaskManager.Task::(_decref) Task=`9dda8ce0-70ae-43da-9fca-35325b9baf21`::ref 1 aborting False >Thread-89::DEBUG::2014-08-11 14:26:28,781::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:26:28,806::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:28,806::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dda8ce0-70ae-43da-9fca-35325b9baf21`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:28,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dda8ce0-70ae-43da-9fca-35325b9baf21`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:26:28,807::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:26:28,807::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:26:28,807::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:26:28,808::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:26:28,808::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:26:28,808::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:26:28,808::task::993::Storage.TaskManager.Task::(_decref) Task=`9dda8ce0-70ae-43da-9fca-35325b9baf21`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:26:30,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:30,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131981 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:32,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:32,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303175 s, 33.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:33,752::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:33,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:33,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:26:34,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:34,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132352 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:36,550::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:36,551::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:26:36,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:26:36,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:36,570::__init__::232::IOProcess::(_processLogs) (5029) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:36,571::__init__::232::IOProcess::(_processLogs) (5029) Queuing response >Thread-16::DEBUG::2014-08-11 14:26:36,572::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000443458 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:36,573::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:36,573::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:36,574::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:36,574::__init__::232::IOProcess::(_processLogs) (5030) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:36,574::__init__::232::IOProcess::(_processLogs) DEBUG|(5030) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:36,576::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:36,576::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:36,576::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:36,577::__init__::232::IOProcess::(_processLogs) (5031) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:36,577::__init__::232::IOProcess::(_processLogs) (5031) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:36,578::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:36,578::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:36,578::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:36,579::__init__::232::IOProcess::(_processLogs) (5032) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:36,579::__init__::232::IOProcess::(_processLogs) DEBUG|(5032) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:36,580::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:36,580::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:36,581::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:36,581::__init__::232::IOProcess::(_processLogs) (5033) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:36,581::__init__::232::IOProcess::(_processLogs) DEBUG|(5033) Queuing response >Dummy-33::DEBUG::2014-08-11 14:26:36,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:36,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132376 s, 77.4 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:26:38,825::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:26:38,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`5baa4229-e0ca-4b60-88ab-151ec81f06bc`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:26:38,827::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:38,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:38,835::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:38,836::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:26:38,840::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:26:38,840::task::1191::Storage.TaskManager.Task::(prepare) Task=`5baa4229-e0ca-4b60-88ab-151ec81f06bc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:26:38,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`5baa4229-e0ca-4b60-88ab-151ec81f06bc`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:26:38,841::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:26:38,841::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:26:38,841::task::993::Storage.TaskManager.Task::(_decref) Task=`5baa4229-e0ca-4b60-88ab-151ec81f06bc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:26:38,851::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:26:38,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`f75ffb40-970f-42a5-9333-ebc97bde1d7c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:26:38,852::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:26:38,853::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c47f72e1-df85-4fc3-b96d-f23fb4d48909`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:26:38,853::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:26:38,854::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:26:38,854::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c47f72e1-df85-4fc3-b96d-f23fb4d48909`::Granted request >Thread-34::DEBUG::2014-08-11 14:26:38,854::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f75ffb40-970f-42a5-9333-ebc97bde1d7c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:26:38,855::task::993::Storage.TaskManager.Task::(_decref) Task=`f75ffb40-970f-42a5-9333-ebc97bde1d7c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:26:38,874::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:38,875::task::1191::Storage.TaskManager.Task::(prepare) Task=`f75ffb40-970f-42a5-9333-ebc97bde1d7c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:38,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`f75ffb40-970f-42a5-9333-ebc97bde1d7c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:26:38,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:26:38,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:26:38,876::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:26:38,876::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:26:38,876::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:26:38,876::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:26:38,877::task::993::Storage.TaskManager.Task::(_decref) Task=`f75ffb40-970f-42a5-9333-ebc97bde1d7c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:26:38,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:38,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013248 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:40,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:40,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131953 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:43,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:43,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132184 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:43,843::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:43,844::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:26:43,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ba81a33-633f-407e-a2dd-fcad65b3aafd`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:26:43,928::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:26:43,930::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000443458', 'lastCheck': '7.3', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:26:43,931::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ba81a33-633f-407e-a2dd-fcad65b3aafd`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000443458', 'lastCheck': '7.3', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:43,931::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:26:43,932::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ba81a33-633f-407e-a2dd-fcad65b3aafd`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:26:43,933::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:26:43,934::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:26:43,936::task::993::Storage.TaskManager.Task::(_decref) Task=`9ba81a33-633f-407e-a2dd-fcad65b3aafd`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:26:43,957::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:26:45,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:45,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132303 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:46,583::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:46,583::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:46,584::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:46,584::__init__::232::IOProcess::(_processLogs) (5034) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:46,584::__init__::232::IOProcess::(_processLogs) (5034) Queuing response >Thread-16::DEBUG::2014-08-11 14:26:46,585::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:26:46,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416803 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:46,605::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:46,605::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:46,605::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:46,605::__init__::232::IOProcess::(_processLogs) (5035) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:46,607::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:46,607::__init__::232::IOProcess::(_processLogs) (5036) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:46,607::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5036) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:46,608::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:46,608::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:46,609::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:46,609::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5037) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:46,609::__init__::232::IOProcess::(_processLogs) (5037) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:46,610::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:46,610::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:46,611::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:46,611::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5038) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:46,611::__init__::232::IOProcess::(_processLogs) (5038) Queuing response >Dummy-33::DEBUG::2014-08-11 14:26:47,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:47,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132742 s, 77.1 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:26:48,890::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:26:48,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`13155c74-544c-425f-80af-1e419655e0fa`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:26:48,926::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:26:48,940::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:26:48,942::task::1191::Storage.TaskManager.Task::(prepare) Task=`13155c74-544c-425f-80af-1e419655e0fa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:48,943::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:26:48,943::task::595::Storage.TaskManager.Task::(_updateState) Task=`13155c74-544c-425f-80af-1e419655e0fa`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:26:48,945::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:26:48,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:26:48,947::task::993::Storage.TaskManager.Task::(_decref) Task=`13155c74-544c-425f-80af-1e419655e0fa`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:48,949::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:48,951::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:26:48,962::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:26:48,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`be3c8be9-2a5a-49e9-b674-55abd6e412ce`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:26:48,963::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:26:48,964::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`051a31d9-8518-4e3f-a961-eb770beb2ce1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:26:48,964::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:26:48,964::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:26:48,965::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`051a31d9-8518-4e3f-a961-eb770beb2ce1`::Granted request >Thread-34::DEBUG::2014-08-11 14:26:48,965::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`be3c8be9-2a5a-49e9-b674-55abd6e412ce`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:26:48,965::task::993::Storage.TaskManager.Task::(_decref) Task=`be3c8be9-2a5a-49e9-b674-55abd6e412ce`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:26:48,985::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:48,986::task::1191::Storage.TaskManager.Task::(prepare) Task=`be3c8be9-2a5a-49e9-b674-55abd6e412ce`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085129728', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:48,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`be3c8be9-2a5a-49e9-b674-55abd6e412ce`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:26:48,986::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:26:48,986::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:26:48,987::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:26:48,987::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:26:48,987::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:26:48,988::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:26:48,988::task::993::Storage.TaskManager.Task::(_decref) Task=`be3c8be9-2a5a-49e9-b674-55abd6e412ce`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:26:49,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:49,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0232005 s, 44.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:51,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:51,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132766 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:26:53,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:53,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131664 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:53,958::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:53,959::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:54,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:26:55,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:55,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133086 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:56,613::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:56,613::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:56,614::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:56,614::__init__::232::IOProcess::(_processLogs) (5039) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:56,615::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5039) Queuing response >Thread-16::DEBUG::2014-08-11 14:26:56,616::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:26:56,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416049 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:26:56,636::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:56,637::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:56,637::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:56,637::__init__::232::IOProcess::(_processLogs) (5040) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:26:56,638::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5040) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:56,639::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:56,640::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:56,640::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:56,640::__init__::232::IOProcess::(_processLogs) (5041) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:56,641::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5041) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:56,641::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:56,642::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:56,642::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:56,642::__init__::232::IOProcess::(_processLogs) (5042) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:56,643::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5042) Queuing response >Thread-14::DEBUG::2014-08-11 14:26:56,644::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:26:56,644::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:26:56,644::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:26:56,645::__init__::232::IOProcess::(_processLogs) (5043) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:26:56,645::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5043) Queuing response >Dummy-33::DEBUG::2014-08-11 14:26:57,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:57,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132318 s, 77.4 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:26:59,001::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:26:59,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2d45a70-cd42-46e5-a911-0c840c7dda7b`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:26:59,028::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:59,035::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:59,038::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:26:59,039::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:26:59,041::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:26:59,041::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2d45a70-cd42-46e5-a911-0c840c7dda7b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:26:59,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2d45a70-cd42-46e5-a911-0c840c7dda7b`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:26:59,042::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:26:59,042::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:26:59,042::task::993::Storage.TaskManager.Task::(_decref) Task=`b2d45a70-cd42-46e5-a911-0c840c7dda7b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:26:59,053::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:26:59,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f830164-6aa0-4ea1-afa0-2f2f977c92be`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:26:59,054::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:26:59,055::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`178c1546-ac10-4817-b68c-b57de54fa095`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:26:59,055::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:26:59,056::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:26:59,056::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`178c1546-ac10-4817-b68c-b57de54fa095`::Granted request >Thread-34::DEBUG::2014-08-11 14:26:59,056::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4f830164-6aa0-4ea1-afa0-2f2f977c92be`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:26:59,057::task::993::Storage.TaskManager.Task::(_decref) Task=`4f830164-6aa0-4ea1-afa0-2f2f977c92be`::ref 1 aborting False >Thread-89::DEBUG::2014-08-11 14:26:59,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0aae1e1-7028-4fdd-b02f-93dab21e66d6`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:26:59,066::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:26:59,068::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416049', 'lastCheck': '2.4', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:26:59,069::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0aae1e1-7028-4fdd-b02f-93dab21e66d6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416049', 'lastCheck': '2.4', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:26:59,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0aae1e1-7028-4fdd-b02f-93dab21e66d6`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:26:59,072::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:26:59,073::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:26:59,074::task::993::Storage.TaskManager.Task::(_decref) Task=`d0aae1e1-7028-4fdd-b02f-93dab21e66d6`::ref 0 aborting False >Thread-34::INFO::2014-08-11 14:26:59,111::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:59,111::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f830164-6aa0-4ea1-afa0-2f2f977c92be`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:26:59,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f830164-6aa0-4ea1-afa0-2f2f977c92be`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:26:59,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:26:59,112::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:26:59,114::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:26:59,116::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:26:59,117::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-89::DEBUG::2014-08-11 14:26:59,118::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:26:59,118::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:26:59,120::task::993::Storage.TaskManager.Task::(_decref) Task=`4f830164-6aa0-4ea1-afa0-2f2f977c92be`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:26:59,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:26:59,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134637 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:01,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:01,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133167 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:03,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:03,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133719 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:04,047::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:04,048::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:04,129::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:27:05,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:05,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132474 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:06,647::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:27:06,647::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:27:06,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:06,648::__init__::232::IOProcess::(_processLogs) (5044) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:06,649::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5044) Queuing response >Thread-16::DEBUG::2014-08-11 14:27:06,650::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:27:06,668::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417088 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:06,670::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:27:06,670::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:27:06,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:06,671::__init__::232::IOProcess::(_processLogs) (5045) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:06,672::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:06,672::__init__::232::IOProcess::(_processLogs) (5046) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:06,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:06,673::__init__::232::IOProcess::(_processLogs) (5047) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:06,674::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:06,674::__init__::232::IOProcess::(_processLogs) (5048) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:27:07,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:07,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132442 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:09,136::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:09,138::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:09,151::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:27:09,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`477215d4-1212-4d8d-af5f-f9b2ec46c9f6`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:09,233::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:09,237::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::INFO::2014-08-11 14:27:09,249::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:09,251::task::1191::Storage.TaskManager.Task::(prepare) Task=`477215d4-1212-4d8d-af5f-f9b2ec46c9f6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:09,252::task::595::Storage.TaskManager.Task::(_updateState) Task=`477215d4-1212-4d8d-af5f-f9b2ec46c9f6`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:09,253::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:09,254::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:09,255::task::993::Storage.TaskManager.Task::(_decref) Task=`477215d4-1212-4d8d-af5f-f9b2ec46c9f6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:27:09,280::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:27:09,283::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b908a5e-0ee6-411b-92cb-01018e2645ee`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:27:09,284::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:27:09,288::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`18ad7f74-c1bc-4293-af0d-3fff5e2e4c3e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:27:09,289::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:27:09,291::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:27:09,292::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`18ad7f74-c1bc-4293-af0d-3fff5e2e4c3e`::Granted request >Thread-34::DEBUG::2014-08-11 14:27:09,295::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3b908a5e-0ee6-411b-92cb-01018e2645ee`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:27:09,298::task::993::Storage.TaskManager.Task::(_decref) Task=`3b908a5e-0ee6-411b-92cb-01018e2645ee`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:27:09,363::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:09,365::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b908a5e-0ee6-411b-92cb-01018e2645ee`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:09,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b908a5e-0ee6-411b-92cb-01018e2645ee`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:27:09,367::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:27:09,369::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:27:09,372::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:27:09,372::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:27:09,372::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:27:09,373::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:27:09,373::task::993::Storage.TaskManager.Task::(_decref) Task=`3b908a5e-0ee6-411b-92cb-01018e2645ee`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:27:09,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:09,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132237 s, 77.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:27:10,235::task::595::Storage.TaskManager.Task::(_updateState) Task=`27a5e559-fce2-43e2-b65f-42a54b297918`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:27:10,236::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:27:10,237::__init__::232::IOProcess::(_processLogs) (5049) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:27:10,238::__init__::232::IOProcess::(_processLogs) (5050) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:27:10,239::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:27:10,239::task::1191::Storage.TaskManager.Task::(prepare) Task=`27a5e559-fce2-43e2-b65f-42a54b297918`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:27:10,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`27a5e559-fce2-43e2-b65f-42a54b297918`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:27:10,240::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:27:10,240::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:27:10,240::task::993::Storage.TaskManager.Task::(_decref) Task=`27a5e559-fce2-43e2-b65f-42a54b297918`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:27:11,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:11,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132702 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:13,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:13,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132057 s, 77.5 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:27:14,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`96395f94-9ed2-41d2-abca-329b61aec088`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:14,227::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:27:14,228::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417088', 'lastCheck': '7.6', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:27:14,229::task::1191::Storage.TaskManager.Task::(prepare) Task=`96395f94-9ed2-41d2-abca-329b61aec088`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417088', 'lastCheck': '7.6', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:27:14,229::task::595::Storage.TaskManager.Task::(_updateState) Task=`96395f94-9ed2-41d2-abca-329b61aec088`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:14,229::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:14,229::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:14,230::task::993::Storage.TaskManager.Task::(_decref) Task=`96395f94-9ed2-41d2-abca-329b61aec088`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:14,244::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:14,246::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:14,247::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:14,249::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:27:15,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:15,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132216 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:16,677::__init__::232::IOProcess::(_processLogs) (5051) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:27:16,678::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:27:16,696::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423733 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:16,698::__init__::232::IOProcess::(_processLogs) (5052) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:16,699::__init__::232::IOProcess::(_processLogs) (5053) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:16,700::__init__::232::IOProcess::(_processLogs) (5054) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:16,701::__init__::232::IOProcess::(_processLogs) (5055) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:27:17,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:17,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131243 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:19,254::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:19,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:19,329::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:19,408::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:27:19,410::task::595::Storage.TaskManager.Task::(_updateState) Task=`606a2f02-f5c9-4333-b2d7-1f5d2701d9c0`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:19,412::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:27:19,426::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:19,427::task::1191::Storage.TaskManager.Task::(prepare) Task=`606a2f02-f5c9-4333-b2d7-1f5d2701d9c0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:19,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`606a2f02-f5c9-4333-b2d7-1f5d2701d9c0`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:19,429::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:19,431::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:19,432::task::993::Storage.TaskManager.Task::(_decref) Task=`606a2f02-f5c9-4333-b2d7-1f5d2701d9c0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:27:19,446::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:27:19,450::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0393cf1-58e5-4265-bb0a-3eb226e1d7bb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:27:19,451::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:27:19,455::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`26b3cd23-8f50-47b0-9883-0345aef3863b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:27:19,456::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:27:19,458::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:27:19,459::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`26b3cd23-8f50-47b0-9883-0345aef3863b`::Granted request >Thread-34::DEBUG::2014-08-11 14:27:19,462::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d0393cf1-58e5-4265-bb0a-3eb226e1d7bb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:27:19,464::task::993::Storage.TaskManager.Task::(_decref) Task=`d0393cf1-58e5-4265-bb0a-3eb226e1d7bb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:27:19,527::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:19,528::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0393cf1-58e5-4265-bb0a-3eb226e1d7bb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:19,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0393cf1-58e5-4265-bb0a-3eb226e1d7bb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:27:19,530::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:27:19,531::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:27:19,533::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:27:19,534::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:27:19,535::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:27:19,536::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:27:19,537::task::993::Storage.TaskManager.Task::(_decref) Task=`d0393cf1-58e5-4265-bb0a-3eb226e1d7bb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:27:20,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:20,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133791 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:22,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:22,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131658 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:24,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:24,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132016 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:24,335::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:24,336::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:24,337::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:27:26,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:26,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132704 s, 77.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:27:26,702::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:27:26,704::__init__::232::IOProcess::(_processLogs) (5056) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:27:26,712::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:27:26,713::__init__::232::IOProcess::(_processLogs) (5057) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 14:27:26,718::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:27:26,719::__init__::232::IOProcess::(_processLogs) (5058) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 14:27:26,722::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:27:26,723::__init__::232::IOProcess::(_processLogs) (5059) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:26,724::__init__::232::IOProcess::(_processLogs) (5060) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:27:26,725::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 14:27:26,725::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:27:26,725::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:27:26,726::__init__::232::IOProcess::(_processLogs) (5061) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:27:26,727::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:27:26,744::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420616 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:26,746::__init__::232::IOProcess::(_processLogs) (5062) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:26,747::__init__::232::IOProcess::(_processLogs) (5063) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:26,748::__init__::232::IOProcess::(_processLogs) (5064) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:26,749::__init__::232::IOProcess::(_processLogs) (5065) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:27:28,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:28,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132178 s, 77.5 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:27:29,339::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce4695da-fb66-4bae-b541-4cb21e91f552`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:29,339::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:27:29,340::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420616', 'lastCheck': '2.6', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:27:29,340::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce4695da-fb66-4bae-b541-4cb21e91f552`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420616', 'lastCheck': '2.6', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:29,344::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:29,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce4695da-fb66-4bae-b541-4cb21e91f552`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:29,432::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:29,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:29,434::task::993::Storage.TaskManager.Task::(_decref) Task=`ce4695da-fb66-4bae-b541-4cb21e91f552`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:29,437::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:29,444::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:29,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:29,558::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:27:29,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`071d72bf-8101-4778-8565-7e15007d8f1a`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:29,559::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:27:29,576::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:29,578::task::1191::Storage.TaskManager.Task::(prepare) Task=`071d72bf-8101-4778-8565-7e15007d8f1a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:29,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`071d72bf-8101-4778-8565-7e15007d8f1a`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:29,580::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:29,581::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:29,583::task::993::Storage.TaskManager.Task::(_decref) Task=`071d72bf-8101-4778-8565-7e15007d8f1a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:27:29,602::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:27:29,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`fcd447ed-5925-4808-b011-fbad48cbb52d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:27:29,606::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:27:29,610::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1d353b93-8c6b-4908-95d8-a4efba3f8b79`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:27:29,611::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:27:29,613::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:27:29,614::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1d353b93-8c6b-4908-95d8-a4efba3f8b79`::Granted request >Thread-34::DEBUG::2014-08-11 14:27:29,617::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fcd447ed-5925-4808-b011-fbad48cbb52d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:27:29,618::task::993::Storage.TaskManager.Task::(_decref) Task=`fcd447ed-5925-4808-b011-fbad48cbb52d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:27:29,664::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:29,665::task::1191::Storage.TaskManager.Task::(prepare) Task=`fcd447ed-5925-4808-b011-fbad48cbb52d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:29,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`fcd447ed-5925-4808-b011-fbad48cbb52d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:27:29,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:27:29,669::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:27:29,671::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:27:29,673::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:27:29,674::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:27:29,675::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:27:29,677::task::993::Storage.TaskManager.Task::(_decref) Task=`fcd447ed-5925-4808-b011-fbad48cbb52d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:27:30,289::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:30,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133322 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:32,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:32,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.024619 s, 41.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:34,455::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:34,457::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:34,533::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:27:34,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:34,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131156 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:36,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:36,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131413 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:36,751::__init__::232::IOProcess::(_processLogs) (5066) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:27:36,752::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:27:36,771::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432054 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:36,772::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:36,773::__init__::232::IOProcess::(_processLogs) (5067) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:36,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:36,774::__init__::232::IOProcess::(_processLogs) (5068) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:36,774::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5068) Queuing response >Thread-14::DEBUG::2014-08-11 14:27:36,776::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:27:36,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:27:36,776::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:36,776::__init__::232::IOProcess::(_processLogs) (5069) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:36,777::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5069) Queuing response >Thread-14::DEBUG::2014-08-11 14:27:36,777::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:27:36,778::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:27:36,778::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:36,778::__init__::232::IOProcess::(_processLogs) (5070) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:36,778::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5070) Queuing response >Dummy-33::DEBUG::2014-08-11 14:27:38,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:38,755::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132136 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:39,536::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:39,537::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:39,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:39,695::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:27:39,696::task::595::Storage.TaskManager.Task::(_updateState) Task=`2786f308-766c-4fd5-8cd1-27385fe84632`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:39,696::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:27:39,709::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:39,709::task::1191::Storage.TaskManager.Task::(prepare) Task=`2786f308-766c-4fd5-8cd1-27385fe84632`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:39,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`2786f308-766c-4fd5-8cd1-27385fe84632`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:39,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:39,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:39,710::task::993::Storage.TaskManager.Task::(_decref) Task=`2786f308-766c-4fd5-8cd1-27385fe84632`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:27:39,720::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:27:39,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8d7b8c2-1aee-4425-8a26-3ef5ec11014b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:27:39,721::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:27:39,722::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d973839c-b3c3-4962-b8d0-6c70b5caadfb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:27:39,723::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:27:39,725::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:27:39,726::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d973839c-b3c3-4962-b8d0-6c70b5caadfb`::Granted request >Thread-34::DEBUG::2014-08-11 14:27:39,729::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a8d7b8c2-1aee-4425-8a26-3ef5ec11014b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:27:39,730::task::993::Storage.TaskManager.Task::(_decref) Task=`a8d7b8c2-1aee-4425-8a26-3ef5ec11014b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:27:39,779::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:39,781::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8d7b8c2-1aee-4425-8a26-3ef5ec11014b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:39,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8d7b8c2-1aee-4425-8a26-3ef5ec11014b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:27:39,783::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:27:39,785::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:27:39,787::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:27:39,789::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:27:39,791::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:27:39,792::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:27:39,794::task::993::Storage.TaskManager.Task::(_decref) Task=`a8d7b8c2-1aee-4425-8a26-3ef5ec11014b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:27:40,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:40,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131248 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:42,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:42,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131795 s, 77.7 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:27:44,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8af244f-5b72-4358-80b4-a8500a5e71eb`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:44,563::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:27:44,633::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432054', 'lastCheck': '7.9', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:27:44,633::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8af244f-5b72-4358-80b4-a8500a5e71eb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432054', 'lastCheck': '7.9', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:27:44,633::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8af244f-5b72-4358-80b4-a8500a5e71eb`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:44,633::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:44,634::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:44,634::task::993::Storage.TaskManager.Task::(_decref) Task=`b8af244f-5b72-4358-80b4-a8500a5e71eb`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:44,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:44,641::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:44,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:44,652::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:27:44,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:44,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131881 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:46,780::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:27:46,781::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:27:46,781::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:46,782::__init__::232::IOProcess::(_processLogs) (5071) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:46,782::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5071) Queuing response >Thread-16::DEBUG::2014-08-11 14:27:46,784::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:27:46,802::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423868 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:46,804::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:27:46,804::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:27:46,804::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:46,805::__init__::232::IOProcess::(_processLogs) (5072) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:46,806::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:46,806::__init__::232::IOProcess::(_processLogs) (5073) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:46,807::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:46,807::__init__::232::IOProcess::(_processLogs) (5074) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:46,808::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:46,809::__init__::232::IOProcess::(_processLogs) (5075) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:46,809::__init__::232::IOProcess::(_processLogs) (5075) Queuing response >Dummy-33::DEBUG::2014-08-11 14:27:46,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:46,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139296 s, 73.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:48,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:49,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132565 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:49,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:49,651::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:49,733::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:49,808::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:27:49,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`937094b6-da22-4da7-901d-5c4add539d19`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:49,810::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:27:49,823::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:49,823::task::1191::Storage.TaskManager.Task::(prepare) Task=`937094b6-da22-4da7-901d-5c4add539d19`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:49,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`937094b6-da22-4da7-901d-5c4add539d19`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:49,824::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:49,824::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:49,824::task::993::Storage.TaskManager.Task::(_decref) Task=`937094b6-da22-4da7-901d-5c4add539d19`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:27:49,834::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:27:49,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bdc530e-e242-4751-a96b-f120d1f3ab85`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:27:49,835::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:27:49,836::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1722c81e-21fd-42d9-b5dc-22f1b0e963dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:27:49,836::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:27:49,836::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:27:49,837::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1722c81e-21fd-42d9-b5dc-22f1b0e963dd`::Granted request >Thread-34::DEBUG::2014-08-11 14:27:49,837::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4bdc530e-e242-4751-a96b-f120d1f3ab85`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:27:49,837::task::993::Storage.TaskManager.Task::(_decref) Task=`4bdc530e-e242-4751-a96b-f120d1f3ab85`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:27:49,857::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:49,858::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bdc530e-e242-4751-a96b-f120d1f3ab85`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:49,858::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bdc530e-e242-4751-a96b-f120d1f3ab85`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:27:49,858::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:27:49,858::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:27:49,859::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:27:49,859::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:27:49,859::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:27:49,859::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:27:49,860::task::993::Storage.TaskManager.Task::(_decref) Task=`4bdc530e-e242-4751-a96b-f120d1f3ab85`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:27:51,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:51,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133088 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:53,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:53,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131747 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:54,740::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:54,742::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:54,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:27:55,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:55,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131996 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:56,811::__init__::232::IOProcess::(_processLogs) (5076) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:27:56,812::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:27:56,830::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418224 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:27:56,831::__init__::232::IOProcess::(_processLogs) (5077) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:27:56,833::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:56,833::__init__::232::IOProcess::(_processLogs) (5078) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:56,834::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:56,834::__init__::232::IOProcess::(_processLogs) (5079) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:56,835::__init__::232::IOProcess::(_processLogs) (5079) Queuing response >Thread-14::DEBUG::2014-08-11 14:27:56,836::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:27:56,836::__init__::232::IOProcess::(_processLogs) (5080) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:27:56,837::__init__::232::IOProcess::(_processLogs) (5080) Queuing response >Dummy-33::DEBUG::2014-08-11 14:27:57,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:57,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132084 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:27:59,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:27:59,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134475 s, 76.1 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:27:59,745::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bd57e8f-92cd-4231-8632-705af9c9498e`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:59,746::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:27:59,833::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418224', 'lastCheck': '3.0', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:27:59,833::task::1191::Storage.TaskManager.Task::(prepare) Task=`0bd57e8f-92cd-4231-8632-705af9c9498e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418224', 'lastCheck': '3.0', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:27:59,833::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bd57e8f-92cd-4231-8632-705af9c9498e`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:59,833::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:59,834::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:59,834::task::993::Storage.TaskManager.Task::(_decref) Task=`0bd57e8f-92cd-4231-8632-705af9c9498e`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:59,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:59,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:27:59,842::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:59,852::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:27:59,873::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-89::DEBUG::2014-08-11 14:27:59,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`d80905bb-6308-46dd-8786-cdcbfe5b13ea`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:27:59,874::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:27:59,887::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:59,887::task::1191::Storage.TaskManager.Task::(prepare) Task=`d80905bb-6308-46dd-8786-cdcbfe5b13ea`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:27:59,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`d80905bb-6308-46dd-8786-cdcbfe5b13ea`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:27:59,887::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:27:59,888::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:27:59,888::task::993::Storage.TaskManager.Task::(_decref) Task=`d80905bb-6308-46dd-8786-cdcbfe5b13ea`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:27:59,897::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 14:27:59,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`122b63df-6574-447d-a998-d00639ac222d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:27:59,898::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:27:59,899::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7e5d5258-14fd-4663-818c-36cfcedf9d12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:27:59,899::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:27:59,899::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:27:59,900::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7e5d5258-14fd-4663-818c-36cfcedf9d12`::Granted request >Thread-34::DEBUG::2014-08-11 14:27:59,900::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`122b63df-6574-447d-a998-d00639ac222d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:27:59,901::task::993::Storage.TaskManager.Task::(_decref) Task=`122b63df-6574-447d-a998-d00639ac222d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:27:59,920::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:59,921::task::1191::Storage.TaskManager.Task::(prepare) Task=`122b63df-6574-447d-a998-d00639ac222d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:27:59,921::task::595::Storage.TaskManager.Task::(_updateState) Task=`122b63df-6574-447d-a998-d00639ac222d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:27:59,921::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:27:59,921::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:27:59,922::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:27:59,922::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:27:59,922::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:27:59,923::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:27:59,923::task::993::Storage.TaskManager.Task::(_decref) Task=`122b63df-6574-447d-a998-d00639ac222d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:28:01,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:01,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132764 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:03,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:03,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131114 s, 78.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:04,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:04,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:04,850::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:28:05,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:05,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131324 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:06,838::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:06,839::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:06,839::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:06,839::__init__::232::IOProcess::(_processLogs) (5081) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:06,840::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|(5081) Queuing response >Thread-16::DEBUG::2014-08-11 14:28:06,841::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:28:06,859::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417163 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:06,861::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:06,861::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:06,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:06,861::__init__::232::IOProcess::(_processLogs) (5082) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:06,862::__init__::232::IOProcess::(_processLogs) (5082) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:06,863::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:06,863::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:06,864::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:06,864::__init__::232::IOProcess::(_processLogs) (5083) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:06,864::__init__::232::IOProcess::(_processLogs) (5083) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:06,865::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:06,865::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:06,866::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:06,866::__init__::232::IOProcess::(_processLogs) (5084) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:06,867::__init__::232::IOProcess::(_processLogs) (5084) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:06,867::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:06,867::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:06,867::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:06,868::__init__::232::IOProcess::(_processLogs) (5085) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:06,868::__init__::232::IOProcess::(_processLogs) (5085) Queuing response >Dummy-33::DEBUG::2014-08-11 14:28:07,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:07,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131776 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:09,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:09,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132067 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:09,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:09,858::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:09,935::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:28:09,936::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:28:09,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cf07f1a-9e9d-4ece-98d0-5b0684c88140`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:28:09,939::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:28:09,952::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:28:09,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`4cf07f1a-9e9d-4ece-98d0-5b0684c88140`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:28:09,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cf07f1a-9e9d-4ece-98d0-5b0684c88140`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:28:09,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:28:09,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:28:09,953::task::993::Storage.TaskManager.Task::(_decref) Task=`4cf07f1a-9e9d-4ece-98d0-5b0684c88140`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:28:09,963::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:28:09,964::task::595::Storage.TaskManager.Task::(_updateState) Task=`704e8b46-72e3-4df3-95bd-6541916462be`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:28:09,964::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:28:09,965::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`61ed3872-2156-48ae-84f8-0ce4629e8cdc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:28:09,965::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:28:09,965::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:28:09,966::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`61ed3872-2156-48ae-84f8-0ce4629e8cdc`::Granted request >Thread-34::DEBUG::2014-08-11 14:28:09,966::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`704e8b46-72e3-4df3-95bd-6541916462be`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:28:09,966::task::993::Storage.TaskManager.Task::(_decref) Task=`704e8b46-72e3-4df3-95bd-6541916462be`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:28:09,986::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:09,986::task::1191::Storage.TaskManager.Task::(prepare) Task=`704e8b46-72e3-4df3-95bd-6541916462be`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863085064192', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:09,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`704e8b46-72e3-4df3-95bd-6541916462be`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:28:09,987::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:28:09,987::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:28:09,987::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:28:09,988::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:28:09,988::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:28:09,988::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:28:09,989::task::993::Storage.TaskManager.Task::(_decref) Task=`704e8b46-72e3-4df3-95bd-6541916462be`::ref 0 aborting False >Thread-55::DEBUG::2014-08-11 14:28:10,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`4149302d-6abe-47ff-b3a1-c0938139ee46`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:28:10,343::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:28:10,344::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:10,344::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:10,345::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:10,345::__init__::232::IOProcess::(_processLogs) (5086) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:28:10,346::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|(5086) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:10,347::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-55::INFO::2014-08-11 14:28:10,347::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:28:10,347::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-55::DEBUG::2014-08-11 14:28:10,349::task::1191::Storage.TaskManager.Task::(prepare) Task=`4149302d-6abe-47ff-b3a1-c0938139ee46`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:28:10,349::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:28:10,350::task::595::Storage.TaskManager.Task::(_updateState) Task=`4149302d-6abe-47ff-b3a1-c0938139ee46`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:28:10,351::__init__::232::IOProcess::(_processLogs) (5087) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:28:10,351::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:28:10,352::__init__::232::IOProcess::(_processLogs) (5087) Queuing response >Thread-55::DEBUG::2014-08-11 14:28:10,352::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:28:10,353::task::993::Storage.TaskManager.Task::(_decref) Task=`4149302d-6abe-47ff-b3a1-c0938139ee46`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:28:11,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:11,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131712 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:13,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:13,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132365 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:14,942::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:14,943::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:28:14,948::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab960d1f-d3f3-48e7-8d62-50b8efbadb1b`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:28:15,034::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-89::INFO::2014-08-11 14:28:15,036::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417163', 'lastCheck': '8.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:15,037::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:28:15,037::task::1191::Storage.TaskManager.Task::(prepare) Task=`ab960d1f-d3f3-48e7-8d62-50b8efbadb1b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417163', 'lastCheck': '8.2', 'valid': True}} >Thread-89::DEBUG::2014-08-11 14:28:15,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab960d1f-d3f3-48e7-8d62-50b8efbadb1b`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:28:15,040::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:28:15,041::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:28:15,041::task::993::Storage.TaskManager.Task::(_decref) Task=`ab960d1f-d3f3-48e7-8d62-50b8efbadb1b`::ref 0 aborting False >Thread-89::DEBUG::2014-08-11 14:28:15,063::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:28:15,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:15,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131894 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:16,870::__init__::232::IOProcess::(_processLogs) (5080) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:16,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:28:16,872::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:28:16,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:16,890::__init__::232::IOProcess::(_processLogs) (5088) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:16,891::__init__::232::IOProcess::(_processLogs) (5088) Queuing response >Thread-16::DEBUG::2014-08-11 14:28:16,892::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000630572 s, 722 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:16,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:16,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:16,893::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:16,894::__init__::232::IOProcess::(_processLogs) (5089) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:16,894::__init__::232::IOProcess::(_processLogs) DEBUG|(5089) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:16,896::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:16,896::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:16,897::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:16,897::__init__::232::IOProcess::(_processLogs) (5090) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:16,897::__init__::232::IOProcess::(_processLogs) (5090) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:16,898::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:16,898::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:16,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:16,899::__init__::232::IOProcess::(_processLogs) (5091) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:16,899::__init__::232::IOProcess::(_processLogs) DEBUG|(5091) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:16,900::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:16,900::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:16,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:16,902::__init__::232::IOProcess::(_processLogs) (5092) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:16,902::__init__::232::IOProcess::(_processLogs) (5092) Queuing response >Dummy-33::DEBUG::2014-08-11 14:28:17,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:17,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131953 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:19,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:19,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013265 s, 77.2 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:28:20,000::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-89::DEBUG::2014-08-11 14:28:20,001::task::595::Storage.TaskManager.Task::(_updateState) Task=`23de4300-d004-47ba-b6c4-adbccd1b4780`::moving from state init -> state preparing >Thread-89::INFO::2014-08-11 14:28:20,032::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-89::INFO::2014-08-11 14:28:20,045::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:28:20,047::task::1191::Storage.TaskManager.Task::(prepare) Task=`23de4300-d004-47ba-b6c4-adbccd1b4780`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:28:20,048::task::595::Storage.TaskManager.Task::(_updateState) Task=`23de4300-d004-47ba-b6c4-adbccd1b4780`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:20,048::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-89::DEBUG::2014-08-11 14:28:20,049::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:28:20,050::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:28:20,052::task::993::Storage.TaskManager.Task::(_decref) Task=`23de4300-d004-47ba-b6c4-adbccd1b4780`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:20,054::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:20,056::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:28:20,065::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:28:20,066::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bc0d7c5-9772-4c0c-9ab2-5d947296ee9f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:28:20,066::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:28:20,067::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`555b804b-ac79-42d4-b34d-6a1e50070e51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:28:20,067::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:28:20,068::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:28:20,068::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`555b804b-ac79-42d4-b34d-6a1e50070e51`::Granted request >Thread-34::DEBUG::2014-08-11 14:28:20,068::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5bc0d7c5-9772-4c0c-9ab2-5d947296ee9f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:28:20,069::task::993::Storage.TaskManager.Task::(_decref) Task=`5bc0d7c5-9772-4c0c-9ab2-5d947296ee9f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:28:20,088::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084998656', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:20,089::task::1191::Storage.TaskManager.Task::(prepare) Task=`5bc0d7c5-9772-4c0c-9ab2-5d947296ee9f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084998656', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:20,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bc0d7c5-9772-4c0c-9ab2-5d947296ee9f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:28:20,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:28:20,090::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:28:20,090::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:28:20,090::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:28:20,090::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:28:20,091::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:28:20,091::task::993::Storage.TaskManager.Task::(_decref) Task=`5bc0d7c5-9772-4c0c-9ab2-5d947296ee9f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:28:21,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:21,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132959 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:23,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:23,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130751 s, 78.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:25,062::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:25,063::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:25,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:28:26,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:26,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132178 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:26,903::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:26,903::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:26,904::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:28:26,905::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:28:26,906::__init__::232::IOProcess::(_processLogs) (5093) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:26,924::__init__::232::IOProcess::(_processLogs) (5093) Queuing response >Thread-16::DEBUG::2014-08-11 14:28:26,925::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000474732 s, 958 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:26,926::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:26,926::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:26,926::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:26,927::__init__::232::IOProcess::(_processLogs) (5094) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:26,927::__init__::232::IOProcess::(_processLogs) DEBUG|(5094) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:26,929::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:26,929::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:26,930::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:26,930::__init__::232::IOProcess::(_processLogs) (5095) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:26,931::__init__::232::IOProcess::(_processLogs) (5095) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:26,931::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:26,932::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:26,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:26,932::__init__::232::IOProcess::(_processLogs) (5096) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:26,933::__init__::232::IOProcess::(_processLogs) (5096) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:26,933::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:26,934::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:26,935::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:26,935::__init__::232::IOProcess::(_processLogs) (5097) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:26,935::__init__::232::IOProcess::(_processLogs) (5097) Queuing response >Dummy-33::DEBUG::2014-08-11 14:28:28,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:28,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132456 s, 77.3 MB/s\n'; <rc> = 0 >Thread-89::DEBUG::2014-08-11 14:28:30,103::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-89::DEBUG::2014-08-11 14:28:30,104::task::595::Storage.TaskManager.Task::(_updateState) Task=`0718246d-ea73-4611-b56e-ca385dca55ce`::moving from state init -> state preparing >Dummy-33::DEBUG::2014-08-11 14:28:30,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-89::INFO::2014-08-11 14:28:30,133::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:30,152::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:30,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:30,155::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:28:30,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134571 s, 76.1 MB/s\n'; <rc> = 0 >Thread-89::INFO::2014-08-11 14:28:30,172::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:28:30,173::task::1191::Storage.TaskManager.Task::(prepare) Task=`0718246d-ea73-4611-b56e-ca385dca55ce`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-89::DEBUG::2014-08-11 14:28:30,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`0718246d-ea73-4611-b56e-ca385dca55ce`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-08-11 14:28:30,176::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-08-11 14:28:30,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-08-11 14:28:30,178::task::993::Storage.TaskManager.Task::(_decref) Task=`0718246d-ea73-4611-b56e-ca385dca55ce`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:28:30,193::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 14:28:30,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`1df720ba-2e32-4023-98a3-3ce147cd10f9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:28:30,197::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:28:30,202::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`968fe4f6-90bd-44c2-99d2-aa34f4f34dd5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:28:30,203::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:28:30,205::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:28:30,207::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`968fe4f6-90bd-44c2-99d2-aa34f4f34dd5`::Granted request >Thread-34::DEBUG::2014-08-11 14:28:30,212::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1df720ba-2e32-4023-98a3-3ce147cd10f9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:28:30,214::task::993::Storage.TaskManager.Task::(_decref) Task=`1df720ba-2e32-4023-98a3-3ce147cd10f9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:28:30,310::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:30,314::task::1191::Storage.TaskManager.Task::(prepare) Task=`1df720ba-2e32-4023-98a3-3ce147cd10f9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:30,316::task::595::Storage.TaskManager.Task::(_updateState) Task=`1df720ba-2e32-4023-98a3-3ce147cd10f9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:28:30,318::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:28:30,320::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:28:30,320::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:28:30,321::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:28:30,321::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:28:30,321::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:28:30,322::task::993::Storage.TaskManager.Task::(_decref) Task=`1df720ba-2e32-4023-98a3-3ce147cd10f9`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 14:28:30,342::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:44824 >Detector thread::DEBUG::2014-08-11 14:28:30,345::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:44824 >Detector thread::DEBUG::2014-08-11 14:28:30,347::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:44824 >Detector thread::DEBUG::2014-08-11 14:28:30,348::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 44824) >Thread-92::DEBUG::2014-08-11 14:28:30,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb8aae44-c116-446d-861f-fce1b42f91f9`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:28:30,366::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:28:30,368::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000474732', 'lastCheck': '3.4', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:28:30,370::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb8aae44-c116-446d-861f-fce1b42f91f9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000474732', 'lastCheck': '3.4', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:28:30,371::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb8aae44-c116-446d-861f-fce1b42f91f9`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:28:30,372::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:28:30,373::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:28:30,374::task::993::Storage.TaskManager.Task::(_decref) Task=`fb8aae44-c116-446d-861f-fce1b42f91f9`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:28:30,419::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:28:32,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:32,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0302847 s, 33.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:34,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:34,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132334 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:35,159::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:35,161::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:35,233::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:28:36,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:36,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135196 s, 75.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:36,936::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:36,936::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:36,937::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:36,937::__init__::232::IOProcess::(_processLogs) (5098) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:36,938::__init__::232::IOProcess::(_processLogs) DEBUG|(5098) Queuing response >Thread-16::DEBUG::2014-08-11 14:28:36,939::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:28:36,958::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412168 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:36,959::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:36,959::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:36,960::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:36,960::__init__::232::IOProcess::(_processLogs) (5099) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:36,961::__init__::232::IOProcess::(_processLogs) DEBUG|(5099) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:36,962::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:36,962::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:36,963::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:36,963::__init__::232::IOProcess::(_processLogs) (5100) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:36,964::__init__::232::IOProcess::(_processLogs) (5100) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:36,964::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:36,965::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:36,965::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:36,965::__init__::232::IOProcess::(_processLogs) (5101) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:36,966::__init__::232::IOProcess::(_processLogs) (5101) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:36,966::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:36,967::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:36,968::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:36,968::__init__::232::IOProcess::(_processLogs) (5102) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:36,968::__init__::232::IOProcess::(_processLogs) (5102) Queuing response >Dummy-33::DEBUG::2014-08-11 14:28:38,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:38,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131749 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:40,240::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:40,241::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:40,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:28:40,348::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:28:40,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc0fcdaa-1972-4f4f-870b-af40814f4eb7`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:28:40,349::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-92::INFO::2014-08-11 14:28:40,362::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:28:40,363::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc0fcdaa-1972-4f4f-870b-af40814f4eb7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:28:40,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc0fcdaa-1972-4f4f-870b-af40814f4eb7`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:28:40,363::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:28:40,363::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:28:40,364::task::993::Storage.TaskManager.Task::(_decref) Task=`dc0fcdaa-1972-4f4f-870b-af40814f4eb7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:28:40,375::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:28:40,375::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6ed9df4-b319-4ee6-a870-a0ed5ac08e02`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:28:40,376::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:28:40,376::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8efc491a-43f8-42fd-8ffe-ef9d70ab5a08`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:28:40,376::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:28:40,377::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:28:40,377::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8efc491a-43f8-42fd-8ffe-ef9d70ab5a08`::Granted request >Thread-34::DEBUG::2014-08-11 14:28:40,378::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e6ed9df4-b319-4ee6-a870-a0ed5ac08e02`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:28:40,378::task::993::Storage.TaskManager.Task::(_decref) Task=`e6ed9df4-b319-4ee6-a870-a0ed5ac08e02`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:28:40,398::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:40,398::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6ed9df4-b319-4ee6-a870-a0ed5ac08e02`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:40,398::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6ed9df4-b319-4ee6-a870-a0ed5ac08e02`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:28:40,398::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:28:40,399::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:28:40,399::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:28:40,399::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:28:40,400::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:28:40,400::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:28:40,400::task::993::Storage.TaskManager.Task::(_decref) Task=`e6ed9df4-b319-4ee6-a870-a0ed5ac08e02`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:28:40,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:40,826::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132288 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:42,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:42,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131522 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:44,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:44,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132706 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:45,341::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:45,342::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:45,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:28:45,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d41a849-b81c-412f-af7f-f64c3b474e56`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:28:45,534::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:28:45,534::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412168', 'lastCheck': '8.6', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:28:45,534::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d41a849-b81c-412f-af7f-f64c3b474e56`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412168', 'lastCheck': '8.6', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:28:45,535::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d41a849-b81c-412f-af7f-f64c3b474e56`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:28:45,535::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:28:45,535::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:28:45,535::task::993::Storage.TaskManager.Task::(_decref) Task=`4d41a849-b81c-412f-af7f-f64c3b474e56`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:28:45,553::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:28:46,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:28:46,979::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:46,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:46,981::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:46,982::__init__::232::IOProcess::(_processLogs) (5103) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:46,988::__init__::232::IOProcess::(_processLogs) DEBUG|(5103) Queuing response >Thread-16::DEBUG::2014-08-11 14:28:46,989::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:28:47,009::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000536797 s, 848 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:47,012::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:47,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:47,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:47,013::__init__::232::IOProcess::(_processLogs) (5104) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:47,015::__init__::232::IOProcess::(_processLogs) DEBUG|(5104) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:47,018::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:47,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:47,018::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:47,019::__init__::232::IOProcess::(_processLogs) (5105) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:47,019::__init__::232::IOProcess::(_processLogs) DEBUG|(5105) Queuing response >Dummy-33::DEBUG::2014-08-11 14:28:47,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0447452 s, 22.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:47,026::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:47,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:47,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:47,029::__init__::232::IOProcess::(_processLogs) (5106) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:47,030::__init__::232::IOProcess::(_processLogs) (5106) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:47,032::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:47,034::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:47,035::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:47,036::__init__::232::IOProcess::(_processLogs) (5107) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:47,037::__init__::232::IOProcess::(_processLogs) (5107) Queuing response >Dummy-33::DEBUG::2014-08-11 14:28:49,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:49,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132796 s, 77.1 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:28:50,412::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-92::DEBUG::2014-08-11 14:28:50,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`c85391b9-e253-4f76-b1d8-3b7e118eb586`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:28:50,432::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:50,440::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:50,442::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:50,443::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:28:50,446::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:28:50,447::task::1191::Storage.TaskManager.Task::(prepare) Task=`c85391b9-e253-4f76-b1d8-3b7e118eb586`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:28:50,447::task::595::Storage.TaskManager.Task::(_updateState) Task=`c85391b9-e253-4f76-b1d8-3b7e118eb586`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:28:50,447::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:28:50,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:28:50,448::task::993::Storage.TaskManager.Task::(_decref) Task=`c85391b9-e253-4f76-b1d8-3b7e118eb586`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:28:50,459::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-34::DEBUG::2014-08-11 14:28:50,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3ed5a08-f8bb-46f0-bfce-caf85473aa31`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:28:50,460::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:28:50,461::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`efdb5d98-acce-404c-94b0-d82244aa52f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:28:50,461::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:28:50,461::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:28:50,462::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`efdb5d98-acce-404c-94b0-d82244aa52f1`::Granted request >Thread-34::DEBUG::2014-08-11 14:28:50,462::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f3ed5a08-f8bb-46f0-bfce-caf85473aa31`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:28:50,462::task::993::Storage.TaskManager.Task::(_decref) Task=`f3ed5a08-f8bb-46f0-bfce-caf85473aa31`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:28:50,482::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:50,482::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3ed5a08-f8bb-46f0-bfce-caf85473aa31`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:28:50,483::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3ed5a08-f8bb-46f0-bfce-caf85473aa31`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:28:50,483::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:28:50,483::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:28:50,484::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:28:50,484::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:28:50,484::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:28:50,484::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:28:50,485::task::993::Storage.TaskManager.Task::(_decref) Task=`f3ed5a08-f8bb-46f0-bfce-caf85473aa31`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:28:51,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:51,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133256 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:53,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:53,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132562 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:55,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:55,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131869 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:55,448::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:55,450::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:28:55,534::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:28:57,035::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:57,035::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:57,035::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:57,036::__init__::232::IOProcess::(_processLogs) DEBUG|(5108) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:57,037::__init__::232::IOProcess::(_processLogs) (5108) Queuing response >Thread-16::DEBUG::2014-08-11 14:28:57,038::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:28:57,057::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416248 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:28:57,059::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:57,059::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:57,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:57,060::__init__::232::IOProcess::(_processLogs) (5109) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:28:57,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:57,061::__init__::232::IOProcess::(_processLogs) (5110) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:57,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:57,062::__init__::232::IOProcess::(_processLogs) (5111) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:57,063::__init__::232::IOProcess::(_processLogs) (5111) Queuing response >Thread-14::DEBUG::2014-08-11 14:28:57,064::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:28:57,064::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:28:57,065::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:28:57,066::__init__::232::IOProcess::(_processLogs) (5112) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:28:57,066::__init__::232::IOProcess::(_processLogs) (5112) Queuing response >Dummy-33::DEBUG::2014-08-11 14:28:57,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:57,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132279 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:28:59,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:28:59,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133206 s, 76.9 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:29:00,496::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:29:00,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`287045c3-3d16-4a21-8030-e52f28572507`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:00,534::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:00,541::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:00,543::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:00,544::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:29:00,546::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:00,547::task::1191::Storage.TaskManager.Task::(prepare) Task=`287045c3-3d16-4a21-8030-e52f28572507`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:00,548::task::595::Storage.TaskManager.Task::(_updateState) Task=`287045c3-3d16-4a21-8030-e52f28572507`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:00,548::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:00,548::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:00,548::task::993::Storage.TaskManager.Task::(_decref) Task=`287045c3-3d16-4a21-8030-e52f28572507`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:29:00,559::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:29:00,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`14795f27-bfb5-4c63-9bdc-e9b710698360`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:29:00,560::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:29:00,560::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dd76ff28-98cb-49a6-80cf-12753eb2d648`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:29:00,561::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:29:00,561::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:29:00,561::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dd76ff28-98cb-49a6-80cf-12753eb2d648`::Granted request >Thread-34::DEBUG::2014-08-11 14:29:00,562::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`14795f27-bfb5-4c63-9bdc-e9b710698360`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:29:00,562::task::993::Storage.TaskManager.Task::(_decref) Task=`14795f27-bfb5-4c63-9bdc-e9b710698360`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:29:00,582::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:00,582::task::1191::Storage.TaskManager.Task::(prepare) Task=`14795f27-bfb5-4c63-9bdc-e9b710698360`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084867584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:00,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`14795f27-bfb5-4c63-9bdc-e9b710698360`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:29:00,583::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:29:00,583::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:29:00,584::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:29:00,584::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:29:00,584::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:29:00,584::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:29:00,585::task::993::Storage.TaskManager.Task::(_decref) Task=`14795f27-bfb5-4c63-9bdc-e9b710698360`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:29:00,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`92d72eae-7255-4079-8545-dfe6d9683239`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:00,648::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:29:00,651::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416248', 'lastCheck': '3.6', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:00,653::task::1191::Storage.TaskManager.Task::(prepare) Task=`92d72eae-7255-4079-8545-dfe6d9683239`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416248', 'lastCheck': '3.6', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:00,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`92d72eae-7255-4079-8545-dfe6d9683239`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:00,656::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:00,657::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:00,659::task::993::Storage.TaskManager.Task::(_decref) Task=`92d72eae-7255-4079-8545-dfe6d9683239`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:29:00,710::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:29:01,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:01,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133628 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:03,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:03,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131933 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:05,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:05,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133667 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:05,552::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:05,556::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:05,560::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:29:07,066::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:07,067::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:07,067::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:07,067::__init__::232::IOProcess::(_processLogs) (5113) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:07,068::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|(5113) Queuing response >Thread-16::DEBUG::2014-08-11 14:29:07,069::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:29:07,088::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000431094 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:07,089::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:07,090::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:07,090::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:07,090::__init__::232::IOProcess::(_processLogs) (5114) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:07,091::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|(5114) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:07,092::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:07,093::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:07,093::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:07,093::__init__::232::IOProcess::(_processLogs) (5115) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:07,094::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|(5115) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:07,094::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:07,095::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:07,095::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:07,095::__init__::232::IOProcess::(_processLogs) (5116) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:07,096::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|(5116) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:07,097::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:07,097::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:07,097::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:07,098::__init__::232::IOProcess::(_processLogs) (5117) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:07,098::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|(5117) Queuing response >Dummy-33::DEBUG::2014-08-11 14:29:07,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:07,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132398 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:09,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:09,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013515 s, 75.8 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:29:10,551::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee33864d-a319-431d-bec8-a829cab55d39`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:29:10,551::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:10,570::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:29:10,597::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:29:10,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf6d4c7a-9e4b-49e5-903b-c534bd54cbf6`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:29:10,638::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-92::INFO::2014-08-11 14:29:10,639::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:10,641::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:29:10,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:10,643::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:10,643::__init__::232::IOProcess::(_processLogs) (5118) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:29:10,644::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|(5118) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:10,646::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:10,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:10,648::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Extracting request information... >Thread-55::INFO::2014-08-11 14:29:10,648::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:29:10,650::__init__::232::IOProcess::(_processLogs) (5119) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:29:10,650::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee33864d-a319-431d-bec8-a829cab55d39`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:29:10,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee33864d-a319-431d-bec8-a829cab55d39`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:29:10,651::__init__::232::IOProcess::(_processLogs) (5119) Queuing response >Thread-55::DEBUG::2014-08-11 14:29:10,652::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:29:10,653::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:29:10,654::task::993::Storage.TaskManager.Task::(_decref) Task=`ee33864d-a319-431d-bec8-a829cab55d39`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:10,655::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:29:10,655::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:10,657::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf6d4c7a-9e4b-49e5-903b-c534bd54cbf6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:10,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf6d4c7a-9e4b-49e5-903b-c534bd54cbf6`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:10,658::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:10,658::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:10,659::task::993::Storage.TaskManager.Task::(_decref) Task=`bf6d4c7a-9e4b-49e5-903b-c534bd54cbf6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:29:10,670::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:29:10,671::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cf279ed-4114-4fb7-965e-b1cae7944f42`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:29:10,671::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:29:10,672::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`da0c5c31-7279-47dc-834b-0bc88f9c9a62`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:29:10,672::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:29:10,673::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:29:10,673::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`da0c5c31-7279-47dc-834b-0bc88f9c9a62`::Granted request >Thread-34::DEBUG::2014-08-11 14:29:10,674::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6cf279ed-4114-4fb7-965e-b1cae7944f42`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:29:10,674::task::993::Storage.TaskManager.Task::(_decref) Task=`6cf279ed-4114-4fb7-965e-b1cae7944f42`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:29:10,694::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:10,695::task::1191::Storage.TaskManager.Task::(prepare) Task=`6cf279ed-4114-4fb7-965e-b1cae7944f42`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:10,695::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cf279ed-4114-4fb7-965e-b1cae7944f42`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:29:10,695::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:29:10,695::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:29:10,696::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:29:10,696::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:29:10,696::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:29:10,696::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:29:10,697::task::993::Storage.TaskManager.Task::(_decref) Task=`6cf279ed-4114-4fb7-965e-b1cae7944f42`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:29:11,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:11,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131711 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:13,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:13,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131243 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:15,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:15,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:15,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:29:15,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:15,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132549 s, 77.3 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:29:15,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0b79424-7f0f-47a6-85b5-98c95aabf817`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:15,829::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:29:15,831::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000431094', 'lastCheck': '8.7', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:15,832::task::1191::Storage.TaskManager.Task::(prepare) Task=`b0b79424-7f0f-47a6-85b5-98c95aabf817`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000431094', 'lastCheck': '8.7', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:15,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0b79424-7f0f-47a6-85b5-98c95aabf817`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:15,835::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:15,836::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:15,837::task::993::Storage.TaskManager.Task::(_decref) Task=`b0b79424-7f0f-47a6-85b5-98c95aabf817`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:29:15,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:29:17,100::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:17,100::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:17,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:17,101::__init__::232::IOProcess::(_processLogs) (5120) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:17,102::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|(5120) Queuing response >Thread-16::DEBUG::2014-08-11 14:29:17,103::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:29:17,121::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414317 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:17,123::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:17,123::__init__::232::IOProcess::(_processLogs) (5111) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:17,123::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:17,124::__init__::232::IOProcess::(_processLogs) (5121) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:17,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:17,126::__init__::232::IOProcess::(_processLogs) (5122) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:17,126::__init__::232::IOProcess::(_processLogs) (5122) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:17,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:17,127::__init__::232::IOProcess::(_processLogs) (5123) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:17,128::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:17,128::__init__::232::IOProcess::(_processLogs) (5124) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:17,129::__init__::232::IOProcess::(_processLogs) (5124) Queuing response >Dummy-33::DEBUG::2014-08-11 14:29:17,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:17,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131554 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:19,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:19,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133894 s, 76.5 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:29:20,709::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:29:20,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e058885-14da-4ece-b2b1-3bf53e555291`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:20,733::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:20,741::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:20,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:20,744::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:29:20,746::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:20,747::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e058885-14da-4ece-b2b1-3bf53e555291`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:20,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e058885-14da-4ece-b2b1-3bf53e555291`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:20,747::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:20,747::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:20,747::task::993::Storage.TaskManager.Task::(_decref) Task=`8e058885-14da-4ece-b2b1-3bf53e555291`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:29:20,758::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:29:20,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a4959f7-bda7-42a9-91d4-b913520ceffd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:29:20,759::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:29:20,760::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`750dcef3-a19f-4bd1-ac1f-a50b84dea99d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:29:20,760::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:29:20,760::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:29:20,760::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`750dcef3-a19f-4bd1-ac1f-a50b84dea99d`::Granted request >Thread-34::DEBUG::2014-08-11 14:29:20,761::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2a4959f7-bda7-42a9-91d4-b913520ceffd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:29:20,761::task::993::Storage.TaskManager.Task::(_decref) Task=`2a4959f7-bda7-42a9-91d4-b913520ceffd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:29:20,781::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:20,781::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a4959f7-bda7-42a9-91d4-b913520ceffd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:20,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a4959f7-bda7-42a9-91d4-b913520ceffd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:29:20,782::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:29:20,782::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:29:20,783::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:29:20,783::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:29:20,783::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:29:20,783::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:29:20,784::task::993::Storage.TaskManager.Task::(_decref) Task=`2a4959f7-bda7-42a9-91d4-b913520ceffd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:29:22,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:22,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131459 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:24,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:24,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132525 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:25,751::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:25,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:25,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:29:26,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:26,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132661 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:27,130::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:27,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:27,132::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:29:27,133::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:29:27,133::__init__::232::IOProcess::(_processLogs) (5125) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:27,151::__init__::232::IOProcess::(_processLogs) (5125) Queuing response >Thread-16::DEBUG::2014-08-11 14:29:27,153::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00044093 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:27,154::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:27,154::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:27,155::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:27,155::__init__::232::IOProcess::(_processLogs) (5126) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:27,156::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|(5126) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:27,157::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:27,158::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:27,158::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:27,159::__init__::232::IOProcess::(_processLogs) (5127) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:27,159::__init__::232::IOProcess::(_processLogs) (5127) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:27,159::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:27,159::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:27,160::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:27,160::__init__::232::IOProcess::(_processLogs) (5128) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:27,161::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|(5128) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:27,161::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:27,162::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:27,162::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:27,162::__init__::232::IOProcess::(_processLogs) (5129) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:27,163::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|(5129) Queuing response >Dummy-33::DEBUG::2014-08-11 14:29:28,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:28,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133648 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:30,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:30,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013603 s, 75.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:30,759::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:30,760::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:29:30,797::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:29:30,836::task::595::Storage.TaskManager.Task::(_updateState) Task=`7116a102-f15f-4a0e-8064-8cd28bff70cc`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:30,837::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:30,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:29:30,851::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:30,852::task::1191::Storage.TaskManager.Task::(prepare) Task=`7116a102-f15f-4a0e-8064-8cd28bff70cc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:30,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`7116a102-f15f-4a0e-8064-8cd28bff70cc`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:30,852::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:30,852::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:30,852::task::993::Storage.TaskManager.Task::(_decref) Task=`7116a102-f15f-4a0e-8064-8cd28bff70cc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:29:30,864::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:29:30,864::task::595::Storage.TaskManager.Task::(_updateState) Task=`de424bb0-5126-42f1-be6c-2db7883b4bdc`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:29:30,865::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:29:30,865::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d78bc5c2-12ac-4431-8df0-c071d33a3de7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:29:30,866::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:29:30,866::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:29:30,866::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d78bc5c2-12ac-4431-8df0-c071d33a3de7`::Granted request >Thread-34::DEBUG::2014-08-11 14:29:30,867::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`de424bb0-5126-42f1-be6c-2db7883b4bdc`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:29:30,867::task::993::Storage.TaskManager.Task::(_decref) Task=`de424bb0-5126-42f1-be6c-2db7883b4bdc`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:29:30,887::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:30,887::task::1191::Storage.TaskManager.Task::(prepare) Task=`de424bb0-5126-42f1-be6c-2db7883b4bdc`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:30,888::task::595::Storage.TaskManager.Task::(_updateState) Task=`de424bb0-5126-42f1-be6c-2db7883b4bdc`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:29:30,888::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:29:30,888::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:29:30,888::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:29:30,889::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:29:30,889::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:29:30,889::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:29:30,889::task::993::Storage.TaskManager.Task::(_decref) Task=`de424bb0-5126-42f1-be6c-2db7883b4bdc`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:29:30,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`610099d7-16a2-4e99-af78-05e8e37c1dd3`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:30,995::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:29:30,997::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00044093', 'lastCheck': '3.8', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:30,998::task::1191::Storage.TaskManager.Task::(prepare) Task=`610099d7-16a2-4e99-af78-05e8e37c1dd3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00044093', 'lastCheck': '3.8', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:30,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`610099d7-16a2-4e99-af78-05e8e37c1dd3`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:31,001::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:31,002::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:31,003::task::993::Storage.TaskManager.Task::(_decref) Task=`610099d7-16a2-4e99-af78-05e8e37c1dd3`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:29:31,047::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:29:32,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:32,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0245004 s, 41.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:34,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:34,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132029 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:35,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:35,848::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:35,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:29:36,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:36,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137198 s, 74.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:37,165::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:37,165::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:37,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:37,166::__init__::232::IOProcess::(_processLogs) (5130) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:37,167::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|(5130) Queuing response >Thread-16::DEBUG::2014-08-11 14:29:37,168::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:29:37,187::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041924 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:37,188::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:37,188::__init__::232::IOProcess::(_processLogs) (5124) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:37,189::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:37,189::__init__::232::IOProcess::(_processLogs) (5131) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:37,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:37,190::__init__::232::IOProcess::(_processLogs) (5132) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:37,191::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:37,191::__init__::232::IOProcess::(_processLogs) (5133) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:37,192::__init__::232::IOProcess::(_processLogs) (5133) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:37,193::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:37,193::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:37,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:37,195::__init__::232::IOProcess::(_processLogs) (5134) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:37,195::__init__::232::IOProcess::(_processLogs) (5134) Queuing response >Dummy-33::DEBUG::2014-08-11 14:29:38,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:38,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132434 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:40,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:40,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132098 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:40,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:40,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:29:40,902::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:29:40,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fba1eb4-16d6-4dd3-a20b-55c0527b6e83`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:40,938::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:40,938::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:29:40,952::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:40,953::task::1191::Storage.TaskManager.Task::(prepare) Task=`3fba1eb4-16d6-4dd3-a20b-55c0527b6e83`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:40,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fba1eb4-16d6-4dd3-a20b-55c0527b6e83`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:40,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:40,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:40,953::task::993::Storage.TaskManager.Task::(_decref) Task=`3fba1eb4-16d6-4dd3-a20b-55c0527b6e83`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:29:40,965::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:29:40,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`32ebcd38-a138-4fad-a373-40a1de72c290`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:29:40,966::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:29:40,967::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`819f142f-af6a-40f8-85d1-dc5bcd1bd217`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:29:40,967::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:29:40,967::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:29:40,967::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`819f142f-af6a-40f8-85d1-dc5bcd1bd217`::Granted request >Thread-34::DEBUG::2014-08-11 14:29:40,968::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`32ebcd38-a138-4fad-a373-40a1de72c290`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:29:40,968::task::993::Storage.TaskManager.Task::(_decref) Task=`32ebcd38-a138-4fad-a373-40a1de72c290`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:29:40,988::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:40,988::task::1191::Storage.TaskManager.Task::(prepare) Task=`32ebcd38-a138-4fad-a373-40a1de72c290`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084802048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:40,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`32ebcd38-a138-4fad-a373-40a1de72c290`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:29:40,989::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:29:40,989::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:29:40,989::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:29:40,990::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:29:40,990::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:29:40,990::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:29:40,990::task::993::Storage.TaskManager.Task::(_decref) Task=`32ebcd38-a138-4fad-a373-40a1de72c290`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:29:42,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:42,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133169 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:44,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:44,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013167 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:45,942::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:45,944::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:46,036::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:29:46,138::task::595::Storage.TaskManager.Task::(_updateState) Task=`08e27a60-ccca-4b3b-96a4-013b977e8985`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:46,138::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:29:46,138::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041924', 'lastCheck': '8.9', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:46,139::task::1191::Storage.TaskManager.Task::(prepare) Task=`08e27a60-ccca-4b3b-96a4-013b977e8985`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041924', 'lastCheck': '8.9', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:29:46,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`08e27a60-ccca-4b3b-96a4-013b977e8985`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:46,139::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:46,139::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:46,139::task::993::Storage.TaskManager.Task::(_decref) Task=`08e27a60-ccca-4b3b-96a4-013b977e8985`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:29:46,159::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:29:46,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:46,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132475 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:47,196::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:47,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:47,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:47,197::__init__::232::IOProcess::(_processLogs) DEBUG|(5135) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:29:47,198::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:29:47,199::__init__::232::IOProcess::(_processLogs) (5135) Queuing response >Thread-16::DEBUG::2014-08-11 14:29:47,218::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000483447 s, 941 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:47,220::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:47,220::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:47,220::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:47,221::__init__::232::IOProcess::(_processLogs) (5136) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:47,221::__init__::232::IOProcess::(_processLogs) DEBUG|(5136) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:47,223::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:47,223::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:47,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:47,224::__init__::232::IOProcess::(_processLogs) (5137) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:47,225::__init__::232::IOProcess::(_processLogs) (5137) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:47,225::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:47,225::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:47,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:47,226::__init__::232::IOProcess::(_processLogs) (5138) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:47,227::__init__::232::IOProcess::(_processLogs) (5138) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:47,227::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:47,228::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:47,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:47,229::__init__::232::IOProcess::(_processLogs) (5139) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:47,229::__init__::232::IOProcess::(_processLogs) (5139) Queuing response >Dummy-33::DEBUG::2014-08-11 14:29:48,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:48,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0175576 s, 58.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:50,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:50,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133208 s, 76.9 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:29:51,004::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:29:51,035::task::595::Storage.TaskManager.Task::(_updateState) Task=`90c36d5c-363c-409a-9224-cf9c13576790`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:29:51,037::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:51,049::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:29:51,052::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:29:51,055::task::1191::Storage.TaskManager.Task::(prepare) Task=`90c36d5c-363c-409a-9224-cf9c13576790`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:51,056::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:29:51,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`90c36d5c-363c-409a-9224-cf9c13576790`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:29:51,058::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:29:51,059::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:29:51,061::task::993::Storage.TaskManager.Task::(_decref) Task=`90c36d5c-363c-409a-9224-cf9c13576790`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:51,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:29:51,089::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:29:51,090::task::595::Storage.TaskManager.Task::(_updateState) Task=`79503e44-ef05-4f48-890c-ba21763aa175`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:29:51,090::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:29:51,091::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7b29c400-b4e8-4649-a143-1ce32c9ae249`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:29:51,091::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:29:51,092::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:29:51,092::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7b29c400-b4e8-4649-a143-1ce32c9ae249`::Granted request >Thread-34::DEBUG::2014-08-11 14:29:51,093::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`79503e44-ef05-4f48-890c-ba21763aa175`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:29:51,093::task::993::Storage.TaskManager.Task::(_decref) Task=`79503e44-ef05-4f48-890c-ba21763aa175`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:29:51,113::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:51,113::task::1191::Storage.TaskManager.Task::(prepare) Task=`79503e44-ef05-4f48-890c-ba21763aa175`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:29:51,113::task::595::Storage.TaskManager.Task::(_updateState) Task=`79503e44-ef05-4f48-890c-ba21763aa175`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:29:51,114::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:29:51,114::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:29:51,114::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:29:51,115::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:29:51,115::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:29:51,115::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:29:51,115::task::993::Storage.TaskManager.Task::(_decref) Task=`79503e44-ef05-4f48-890c-ba21763aa175`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:29:53,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:53,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131817 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:29:55,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:55,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132227 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:56,079::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:56,085::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:29:56,134::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:29:57,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:57,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132218 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:57,231::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:57,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:57,234::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:57,235::__init__::232::IOProcess::(_processLogs) (5140) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:57,237::__init__::232::IOProcess::(_processLogs) DEBUG|(5140) Queuing response >Thread-16::DEBUG::2014-08-11 14:29:57,238::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:29:57,262::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00364306 s, 125 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:29:57,264::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:57,265::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:57,266::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:57,267::__init__::232::IOProcess::(_processLogs) (5141) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:29:57,269::__init__::232::IOProcess::(_processLogs) DEBUG|(5141) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:57,271::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:57,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:57,273::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:57,274::__init__::232::IOProcess::(_processLogs) (5142) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:57,277::__init__::232::IOProcess::(_processLogs) DEBUG|(5142) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:57,279::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:57,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:57,281::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:57,283::__init__::232::IOProcess::(_processLogs) (5143) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:57,285::__init__::232::IOProcess::(_processLogs) DEBUG|(5143) Queuing response >Thread-14::DEBUG::2014-08-11 14:29:57,286::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:29:57,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:29:57,288::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:29:57,289::__init__::232::IOProcess::(_processLogs) (5144) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:29:57,291::__init__::232::IOProcess::(_processLogs) DEBUG|(5144) Queuing response >Dummy-33::DEBUG::2014-08-11 14:29:59,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:29:59,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132064 s, 77.5 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:30:01,128::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:30:01,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`d12cf55c-bef3-49c2-ab7e-f6381eb75747`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:01,135::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:01,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:01,144::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:01,147::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:30:01,149::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:01,149::task::1191::Storage.TaskManager.Task::(prepare) Task=`d12cf55c-bef3-49c2-ab7e-f6381eb75747`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:01,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`d12cf55c-bef3-49c2-ab7e-f6381eb75747`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:01,150::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:01,150::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:01,150::task::993::Storage.TaskManager.Task::(_decref) Task=`d12cf55c-bef3-49c2-ab7e-f6381eb75747`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:30:01,162::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:30:01,163::task::595::Storage.TaskManager.Task::(_updateState) Task=`61b28045-2fb6-4a53-bfed-e10c9b3c1864`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:30:01,163::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:30:01,164::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9dbbe96c-df5e-4098-a33d-e88fec448070`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:30:01,164::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:30:01,164::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:30:01,165::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9dbbe96c-df5e-4098-a33d-e88fec448070`::Granted request >Thread-34::DEBUG::2014-08-11 14:30:01,165::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`61b28045-2fb6-4a53-bfed-e10c9b3c1864`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:30:01,165::task::993::Storage.TaskManager.Task::(_decref) Task=`61b28045-2fb6-4a53-bfed-e10c9b3c1864`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:30:01,186::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:01,186::task::1191::Storage.TaskManager.Task::(prepare) Task=`61b28045-2fb6-4a53-bfed-e10c9b3c1864`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:01,186::task::595::Storage.TaskManager.Task::(_updateState) Task=`61b28045-2fb6-4a53-bfed-e10c9b3c1864`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:30:01,186::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:30:01,187::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:30:01,187::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:30:01,187::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:30:01,188::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:30:01,188::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:30:01,188::task::993::Storage.TaskManager.Task::(_decref) Task=`61b28045-2fb6-4a53-bfed-e10c9b3c1864`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:30:01,248::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cb3e3d8-17c1-44d7-9342-8715e7bcc835`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:01,250::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:30:01,252::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00364306', 'lastCheck': '4.0', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:01,254::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cb3e3d8-17c1-44d7-9342-8715e7bcc835`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00364306', 'lastCheck': '4.0', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:01,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cb3e3d8-17c1-44d7-9342-8715e7bcc835`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:01,256::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:01,257::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:01,258::task::993::Storage.TaskManager.Task::(_decref) Task=`3cb3e3d8-17c1-44d7-9342-8715e7bcc835`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:30:01,308::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:30:01,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:01,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131578 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:03,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:03,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131646 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:05,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:05,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131241 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:06,155::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:06,156::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:06,236::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:30:07,293::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:07,294::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:07,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:07,294::__init__::232::IOProcess::(_processLogs) (5145) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:07,295::__init__::232::IOProcess::(_processLogs) DEBUG|(5145) Queuing response >Thread-16::DEBUG::2014-08-11 14:30:07,296::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:30:07,315::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000451992 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:07,317::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:07,317::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:07,317::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:07,318::__init__::232::IOProcess::(_processLogs) (5146) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:07,318::__init__::232::IOProcess::(_processLogs) DEBUG|(5146) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:07,320::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:07,320::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:07,320::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:07,320::__init__::232::IOProcess::(_processLogs) (5147) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:07,321::__init__::232::IOProcess::(_processLogs) DEBUG|(5147) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:07,322::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:07,323::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:07,323::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:07,324::__init__::232::IOProcess::(_processLogs) (5148) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:07,324::__init__::232::IOProcess::(_processLogs) (5148) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:07,324::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:07,324::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:07,325::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:07,325::__init__::232::IOProcess::(_processLogs) (5149) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:07,326::__init__::232::IOProcess::(_processLogs) DEBUG|(5149) Queuing response >Dummy-33::DEBUG::2014-08-11 14:30:07,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:07,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01318 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:09,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:09,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013124 s, 78.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:30:10,784::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c36a5a6-98e1-4bfe-a4bb-f339e6e41165`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:30:10,784::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:30:10,785::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:10,786::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:10,786::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:10,786::__init__::232::IOProcess::(_processLogs) (5150) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:30:10,787::__init__::232::IOProcess::(_processLogs) DEBUG|(5150) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:10,788::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:10,788::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:30:10,789::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:30:10,790::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:30:10,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c36a5a6-98e1-4bfe-a4bb-f339e6e41165`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:30:10,791::__init__::232::IOProcess::(_processLogs) (5151) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:30:10,791::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c36a5a6-98e1-4bfe-a4bb-f339e6e41165`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:30:10,792::__init__::232::IOProcess::(_processLogs) (5151) Queuing response >Thread-55::DEBUG::2014-08-11 14:30:10,793::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:30:10,794::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:30:10,794::task::993::Storage.TaskManager.Task::(_decref) Task=`6c36a5a6-98e1-4bfe-a4bb-f339e6e41165`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:30:11,201::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:30:11,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`46679ee4-d836-44c5-af8d-2b5c00a17b8f`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:11,202::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-92::INFO::2014-08-11 14:30:11,215::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:11,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`46679ee4-d836-44c5-af8d-2b5c00a17b8f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:11,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`46679ee4-d836-44c5-af8d-2b5c00a17b8f`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:11,216::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:11,216::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:11,216::task::993::Storage.TaskManager.Task::(_decref) Task=`46679ee4-d836-44c5-af8d-2b5c00a17b8f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:30:11,227::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:30:11,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f361f63-7ed5-4223-b69d-5155e37f9e46`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:30:11,228::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:30:11,228::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`23f06c42-41e7-43a5-a4a7-e19448ce0dde`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:30:11,229::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:30:11,229::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:30:11,229::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`23f06c42-41e7-43a5-a4a7-e19448ce0dde`::Granted request >Thread-34::DEBUG::2014-08-11 14:30:11,230::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0f361f63-7ed5-4223-b69d-5155e37f9e46`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:30:11,230::task::993::Storage.TaskManager.Task::(_decref) Task=`0f361f63-7ed5-4223-b69d-5155e37f9e46`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:11,243::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:11,245::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:11,246::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:30:11,250::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:11,250::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f361f63-7ed5-4223-b69d-5155e37f9e46`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:11,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f361f63-7ed5-4223-b69d-5155e37f9e46`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:30:11,251::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:30:11,251::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:30:11,251::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:30:11,252::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:30:11,252::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:30:11,252::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:30:11,252::task::993::Storage.TaskManager.Task::(_decref) Task=`0f361f63-7ed5-4223-b69d-5155e37f9e46`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:30:11,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:11,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131109 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:13,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:13,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013131 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:15,738::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:15,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0263705 s, 38.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:16,253::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:16,254::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:16,336::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:30:16,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fc5e112-58c9-4032-b18c-0565d3fbc8da`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:16,407::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:30:16,408::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000451992', 'lastCheck': '9.1', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:16,408::task::1191::Storage.TaskManager.Task::(prepare) Task=`0fc5e112-58c9-4032-b18c-0565d3fbc8da`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000451992', 'lastCheck': '9.1', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:16,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fc5e112-58c9-4032-b18c-0565d3fbc8da`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:16,408::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:16,409::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:16,409::task::993::Storage.TaskManager.Task::(_decref) Task=`0fc5e112-58c9-4032-b18c-0565d3fbc8da`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:30:16,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:30:17,327::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:17,328::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:17,328::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:30:17,330::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:30:17,330::__init__::232::IOProcess::(_processLogs) (5152) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:17,348::__init__::232::IOProcess::(_processLogs) (5152) Queuing response >Thread-16::DEBUG::2014-08-11 14:30:17,349::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00101571 s, 448 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:17,350::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:17,351::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:17,351::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:17,351::__init__::232::IOProcess::(_processLogs) (5153) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:17,352::__init__::232::IOProcess::(_processLogs) DEBUG|(5153) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:17,353::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:17,354::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:17,355::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:17,355::__init__::232::IOProcess::(_processLogs) (5154) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:17,355::__init__::232::IOProcess::(_processLogs) (5154) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:17,356::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:17,356::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:17,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:17,356::__init__::232::IOProcess::(_processLogs) (5155) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:17,357::__init__::232::IOProcess::(_processLogs) (5155) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:17,358::__init__::232::IOProcess::(_processLogs) (5155) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:17,358::__init__::232::IOProcess::(_processLogs) (5155) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:17,359::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:17,360::__init__::232::IOProcess::(_processLogs) (5156) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:17,360::__init__::232::IOProcess::(_processLogs) (5156) Queuing response >Dummy-33::DEBUG::2014-08-11 14:30:17,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:17,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132639 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:19,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:19,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133337 s, 76.8 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:30:21,263::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:30:21,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7156b5e-206a-4bfd-b043-affabdfcbd36`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:21,335::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:21,343::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:21,345::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:21,346::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:30:21,348::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:21,349::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7156b5e-206a-4bfd-b043-affabdfcbd36`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:21,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7156b5e-206a-4bfd-b043-affabdfcbd36`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:21,349::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:21,349::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:21,349::task::993::Storage.TaskManager.Task::(_decref) Task=`e7156b5e-206a-4bfd-b043-affabdfcbd36`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:30:21,361::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:30:21,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`d65c94c5-d8cf-44ce-8ed8-08c4b4af11bb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:30:21,362::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:30:21,363::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0bd01f01-f26a-494a-9a39-5ed43876b906`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:30:21,363::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:30:21,364::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:30:21,364::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0bd01f01-f26a-494a-9a39-5ed43876b906`::Granted request >Thread-34::DEBUG::2014-08-11 14:30:21,365::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d65c94c5-d8cf-44ce-8ed8-08c4b4af11bb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:30:21,365::task::993::Storage.TaskManager.Task::(_decref) Task=`d65c94c5-d8cf-44ce-8ed8-08c4b4af11bb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:30:21,385::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:21,385::task::1191::Storage.TaskManager.Task::(prepare) Task=`d65c94c5-d8cf-44ce-8ed8-08c4b4af11bb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:21,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`d65c94c5-d8cf-44ce-8ed8-08c4b4af11bb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:30:21,385::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:30:21,386::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:30:21,386::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:30:21,386::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:30:21,387::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:30:21,387::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:30:21,387::task::993::Storage.TaskManager.Task::(_decref) Task=`d65c94c5-d8cf-44ce-8ed8-08c4b4af11bb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:30:21,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:21,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131243 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:23,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:24,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131294 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:26,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:26,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132104 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:26,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:26,355::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:26,437::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:30:27,360::__init__::232::IOProcess::(_processLogs) (5155) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:27,361::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:27,361::__init__::232::IOProcess::(_processLogs) (5155) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:27,361::__init__::232::IOProcess::(_processLogs) (5157) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:27,362::__init__::232::IOProcess::(_processLogs) (5155) Queuing responseDEBUG|(5157) Queuing response >Thread-16::DEBUG::2014-08-11 14:30:27,363::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:30:27,382::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000411538 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:27,384::__init__::232::IOProcess::(_processLogs) (5155) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:27,384::__init__::232::IOProcess::(_processLogs) (5155) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:27,384::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:27,385::__init__::232::IOProcess::(_processLogs) (5158) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:27,386::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:27,386::__init__::232::IOProcess::(_processLogs) (5159) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:27,387::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:27,387::__init__::232::IOProcess::(_processLogs) (5160) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:27,388::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:27,389::__init__::232::IOProcess::(_processLogs) (5161) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:27,389::__init__::232::IOProcess::(_processLogs) (5161) Queuing response >Dummy-33::DEBUG::2014-08-11 14:30:28,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:28,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132711 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:30,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:30,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132449 s, 77.3 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:30:31,399::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:30:31,400::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea8848a7-713c-409c-bfeb-06ac6f54a6e0`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:31,437::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:31,445::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:31,450::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:31,452::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:30:31,452::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:31,454::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea8848a7-713c-409c-bfeb-06ac6f54a6e0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:31,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea8848a7-713c-409c-bfeb-06ac6f54a6e0`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:31,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:31,456::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:31,456::task::993::Storage.TaskManager.Task::(_decref) Task=`ea8848a7-713c-409c-bfeb-06ac6f54a6e0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:30:31,468::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:30:31,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`5229a76e-0985-41ae-8e51-aa545f861b83`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:30:31,469::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:30:31,470::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a377b4b5-3f65-4e17-a5fa-8ddef895698e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:30:31,470::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:30:31,470::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:30:31,471::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a377b4b5-3f65-4e17-a5fa-8ddef895698e`::Granted request >Thread-34::DEBUG::2014-08-11 14:30:31,471::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5229a76e-0985-41ae-8e51-aa545f861b83`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:30:31,472::task::993::Storage.TaskManager.Task::(_decref) Task=`5229a76e-0985-41ae-8e51-aa545f861b83`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:30:31,492::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:31,492::task::1191::Storage.TaskManager.Task::(prepare) Task=`5229a76e-0985-41ae-8e51-aa545f861b83`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084736512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:31,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`5229a76e-0985-41ae-8e51-aa545f861b83`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:30:31,493::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:30:31,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:30:31,493::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:30:31,494::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:30:31,494::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:30:31,494::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:30:31,494::task::993::Storage.TaskManager.Task::(_decref) Task=`5229a76e-0985-41ae-8e51-aa545f861b83`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:30:31,528::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c3c68d3-f9e5-420c-88c2-37803b6ce6a5`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:31,529::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:30:31,531::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411538', 'lastCheck': '4.1', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:31,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c3c68d3-f9e5-420c-88c2-37803b6ce6a5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411538', 'lastCheck': '4.1', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:31,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c3c68d3-f9e5-420c-88c2-37803b6ce6a5`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:31,534::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:31,535::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:31,537::task::993::Storage.TaskManager.Task::(_decref) Task=`8c3c68d3-f9e5-420c-88c2-37803b6ce6a5`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:30:31,581::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:30:32,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:32,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272379 s, 37.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:34,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:34,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141999 s, 72.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:36,462::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:36,463::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:30:36,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:36,556::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:30:36,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133019 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:37,391::__init__::232::IOProcess::(_processLogs) (5161) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:37,392::__init__::232::IOProcess::(_processLogs) (5161) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:37,393::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:30:37,393::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:30:37,394::__init__::232::IOProcess::(_processLogs) (5162) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:37,412::__init__::232::IOProcess::(_processLogs) (5162) Queuing response >Thread-16::DEBUG::2014-08-11 14:30:37,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000436831 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:37,415::__init__::232::IOProcess::(_processLogs) (5161) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:37,415::__init__::232::IOProcess::(_processLogs) (5161) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:37,415::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:37,416::__init__::232::IOProcess::(_processLogs) (5163) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:37,416::__init__::232::IOProcess::(_processLogs) DEBUG|(5163) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:37,418::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:37,418::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:37,419::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:37,419::__init__::232::IOProcess::(_processLogs) (5164) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:37,419::__init__::232::IOProcess::(_processLogs) (5164) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:37,420::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:37,420::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:37,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:37,421::__init__::232::IOProcess::(_processLogs) (5165) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:37,421::__init__::232::IOProcess::(_processLogs) (5165) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:37,421::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:37,422::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:37,423::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:37,423::__init__::232::IOProcess::(_processLogs) (5166) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:37,424::__init__::232::IOProcess::(_processLogs) (5166) Queuing response >Dummy-33::DEBUG::2014-08-11 14:30:38,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:38,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013419 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:40,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:40,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131801 s, 77.7 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:30:41,510::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:30:41,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`c261458f-c064-495c-8ef2-b3fec6be5888`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:41,536::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-92::INFO::2014-08-11 14:30:41,549::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:41,550::task::1191::Storage.TaskManager.Task::(prepare) Task=`c261458f-c064-495c-8ef2-b3fec6be5888`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:41,550::task::595::Storage.TaskManager.Task::(_updateState) Task=`c261458f-c064-495c-8ef2-b3fec6be5888`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:41,550::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:41,551::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:41,551::task::993::Storage.TaskManager.Task::(_decref) Task=`c261458f-c064-495c-8ef2-b3fec6be5888`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:30:41,561::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:30:41,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1d3680f-36fd-43a0-b63e-88c5cdbcb318`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:30:41,564::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:41,565::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:30:41,568::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aa0bf315-d05e-4a0f-bb56-d3972e32dd3e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:30:41,570::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:41,571::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:30:41,572::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:30:41,573::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aa0bf315-d05e-4a0f-bb56-d3972e32dd3e`::Granted request >Thread-34::DEBUG::2014-08-11 14:30:41,576::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d1d3680f-36fd-43a0-b63e-88c5cdbcb318`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:41,577::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:30:41,578::task::993::Storage.TaskManager.Task::(_decref) Task=`d1d3680f-36fd-43a0-b63e-88c5cdbcb318`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:30:41,600::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084670976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:41,601::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1d3680f-36fd-43a0-b63e-88c5cdbcb318`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084670976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:41,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1d3680f-36fd-43a0-b63e-88c5cdbcb318`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:30:41,601::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:30:41,601::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:30:41,602::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:30:41,602::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:30:41,602::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:30:41,602::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:30:41,603::task::993::Storage.TaskManager.Task::(_decref) Task=`d1d3680f-36fd-43a0-b63e-88c5cdbcb318`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:30:42,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:42,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013203 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:44,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:44,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132069 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:46,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:46,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:46,638::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:30:46,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`836f88b4-027b-49e6-a6f0-f4e1bd11b237`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:46,699::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:30:46,700::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436831', 'lastCheck': '9.3', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:46,700::task::1191::Storage.TaskManager.Task::(prepare) Task=`836f88b4-027b-49e6-a6f0-f4e1bd11b237`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436831', 'lastCheck': '9.3', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:30:46,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`836f88b4-027b-49e6-a6f0-f4e1bd11b237`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:46,700::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:46,701::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:46,701::task::993::Storage.TaskManager.Task::(_decref) Task=`836f88b4-027b-49e6-a6f0-f4e1bd11b237`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:30:46,725::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:30:46,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:46,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132131 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:47,424::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:47,424::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:47,425::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:47,425::__init__::232::IOProcess::(_processLogs) (5167) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:47,426::__init__::232::IOProcess::(_processLogs) DEBUG|(5167) Queuing response >Thread-16::DEBUG::2014-08-11 14:30:47,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:30:47,445::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00047666 s, 955 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:47,446::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:47,447::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:47,447::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:47,447::__init__::232::IOProcess::(_processLogs) (5168) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:47,447::__init__::232::IOProcess::(_processLogs) DEBUG|(5168) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:47,448::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:47,449::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:47,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:47,450::__init__::232::IOProcess::(_processLogs) DEBUG|(5169) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:47,450::__init__::232::IOProcess::(_processLogs) (5169) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:47,451::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:47,451::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:47,451::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:47,451::__init__::232::IOProcess::(_processLogs) (5170) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:47,452::__init__::232::IOProcess::(_processLogs) (5170) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:47,452::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:47,453::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:47,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:47,453::__init__::232::IOProcess::(_processLogs) DEBUG|(5171) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:47,453::__init__::232::IOProcess::(_processLogs) (5171) Queuing response >Dummy-33::DEBUG::2014-08-11 14:30:48,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:48,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132532 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:50,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:50,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133699 s, 76.6 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:30:51,617::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:30:51,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`86ca01f1-5ac1-4e53-b721-517f6d99f369`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:30:51,638::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:51,647::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:51,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:51,651::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:30:51,652::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:51,653::task::1191::Storage.TaskManager.Task::(prepare) Task=`86ca01f1-5ac1-4e53-b721-517f6d99f369`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:30:51,653::task::595::Storage.TaskManager.Task::(_updateState) Task=`86ca01f1-5ac1-4e53-b721-517f6d99f369`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:30:51,653::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:30:51,653::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:30:51,654::task::993::Storage.TaskManager.Task::(_decref) Task=`86ca01f1-5ac1-4e53-b721-517f6d99f369`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:30:51,664::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:30:51,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e82cc9a-97d0-49a4-b8e9-3776766dca69`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:30:51,665::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:30:51,666::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cb3afc84-af34-4ddd-baed-e1e3a58303a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:30:51,666::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:30:51,666::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:30:51,666::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cb3afc84-af34-4ddd-baed-e1e3a58303a7`::Granted request >Thread-34::DEBUG::2014-08-11 14:30:51,667::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7e82cc9a-97d0-49a4-b8e9-3776766dca69`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:30:51,667::task::993::Storage.TaskManager.Task::(_decref) Task=`7e82cc9a-97d0-49a4-b8e9-3776766dca69`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:30:51,687::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084670976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:51,688::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e82cc9a-97d0-49a4-b8e9-3776766dca69`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863084670976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:30:51,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e82cc9a-97d0-49a4-b8e9-3776766dca69`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:30:51,688::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:30:51,688::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:30:51,689::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:30:51,689::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:30:51,689::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:30:51,689::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:30:51,690::task::993::Storage.TaskManager.Task::(_decref) Task=`7e82cc9a-97d0-49a4-b8e9-3776766dca69`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:30:52,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:53,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132942 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:30:55,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:55,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132104 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:56,658::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:56,659::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:30:56,738::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:30:57,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:57,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131393 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:57,455::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:57,456::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:57,456::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:57,456::__init__::232::IOProcess::(_processLogs) (5172) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:57,457::__init__::232::IOProcess::(_processLogs) DEBUG|(5172) Queuing response >Thread-16::DEBUG::2014-08-11 14:30:57,457::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:30:57,476::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0004214 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:30:57,477::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:57,478::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:57,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:57,478::__init__::232::IOProcess::(_processLogs) (5173) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:30:57,479::__init__::232::IOProcess::(_processLogs) (5173) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:57,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:57,481::__init__::232::IOProcess::(_processLogs) (5174) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:57,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:57,482::__init__::232::IOProcess::(_processLogs) (5175) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:57,482::__init__::232::IOProcess::(_processLogs) (5175) Queuing response >Thread-14::DEBUG::2014-08-11 14:30:57,483::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:30:57,484::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:30:57,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:30:57,485::__init__::232::IOProcess::(_processLogs) (5176) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:30:57,485::__init__::232::IOProcess::(_processLogs) (5176) Queuing response >Dummy-33::DEBUG::2014-08-11 14:30:59,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:30:59,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132516 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:01,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:01,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133571 s, 76.7 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:31:01,702::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:31:01,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`a89c019b-8a51-4906-8713-c786549330e4`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:01,739::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:01,746::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:01,748::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:01,749::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:31:01,751::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:01,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`a89c019b-8a51-4906-8713-c786549330e4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:01,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`a89c019b-8a51-4906-8713-c786549330e4`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:01,752::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:01,752::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:01,753::task::993::Storage.TaskManager.Task::(_decref) Task=`a89c019b-8a51-4906-8713-c786549330e4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:31:01,764::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:31:01,764::task::595::Storage.TaskManager.Task::(_updateState) Task=`81c92570-2a5b-412f-afb0-ff52cb1e3a04`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:31:01,765::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:31:01,765::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4959e67-8d5b-4373-b14c-f9604615eb83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:31:01,766::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:31:01,766::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:31:01,766::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4959e67-8d5b-4373-b14c-f9604615eb83`::Granted request >Thread-34::DEBUG::2014-08-11 14:31:01,767::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`81c92570-2a5b-412f-afb0-ff52cb1e3a04`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:31:01,767::task::993::Storage.TaskManager.Task::(_decref) Task=`81c92570-2a5b-412f-afb0-ff52cb1e3a04`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:31:01,787::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082508288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:01,788::task::1191::Storage.TaskManager.Task::(prepare) Task=`81c92570-2a5b-412f-afb0-ff52cb1e3a04`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082508288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:01,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`81c92570-2a5b-412f-afb0-ff52cb1e3a04`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:31:01,788::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:31:01,788::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:31:01,789::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:31:01,789::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:31:01,789::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:31:01,789::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:31:01,790::task::993::Storage.TaskManager.Task::(_decref) Task=`81c92570-2a5b-412f-afb0-ff52cb1e3a04`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:31:01,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae845db0-466e-41d6-81b1-7b78d9188db9`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:01,821::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:31:01,822::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0004214', 'lastCheck': '4.3', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:31:01,824::task::1191::Storage.TaskManager.Task::(prepare) Task=`ae845db0-466e-41d6-81b1-7b78d9188db9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0004214', 'lastCheck': '4.3', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:31:01,825::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae845db0-466e-41d6-81b1-7b78d9188db9`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:01,826::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:01,827::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:01,829::task::993::Storage.TaskManager.Task::(_decref) Task=`ae845db0-466e-41d6-81b1-7b78d9188db9`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:31:01,883::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:31:03,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:03,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01327 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:05,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:05,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013365 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:06,756::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:06,758::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:06,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:31:07,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:07,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132237 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:31:07,486::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:07,486::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:07,487::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:07,487::__init__::232::IOProcess::(_processLogs) (5177) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:07,487::__init__::232::IOProcess::(_processLogs) (5177) Queuing response >Thread-16::DEBUG::2014-08-11 14:31:07,488::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:31:07,506::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423841 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:31:07,507::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:07,508::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:07,508::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:07,508::__init__::232::IOProcess::(_processLogs) (5178) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:07,509::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|(5178) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:07,510::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:07,511::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:07,511::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:07,511::__init__::232::IOProcess::(_processLogs) (5179) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:07,512::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|(5179) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:07,512::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:07,513::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:07,514::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:07,514::__init__::232::IOProcess::(_processLogs) (5180) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:07,514::__init__::232::IOProcess::(_processLogs) (5180) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:07,515::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:07,515::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:07,515::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:07,515::__init__::232::IOProcess::(_processLogs) (5181) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:07,516::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|(5181) Queuing response >Dummy-33::DEBUG::2014-08-11 14:31:09,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:09,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133199 s, 76.9 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:31:10,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`83bb4ac7-e4cb-4b57-848c-159f3af5e302`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:31:10,904::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:31:10,905::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:10,905::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:10,905::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:10,905::__init__::232::IOProcess::(_processLogs) (5182) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:31:10,906::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|(5182) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:10,907::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:10,907::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:31:10,908::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:31:10,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:10,910::__init__::232::IOProcess::(_processLogs) (5183) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:31:10,910::task::1191::Storage.TaskManager.Task::(prepare) Task=`83bb4ac7-e4cb-4b57-848c-159f3af5e302`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:31:10,911::__init__::232::IOProcess::(_processLogs) (5183) Queuing response >Thread-55::DEBUG::2014-08-11 14:31:10,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`83bb4ac7-e4cb-4b57-848c-159f3af5e302`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:31:10,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:31:10,912::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:31:10,913::task::993::Storage.TaskManager.Task::(_decref) Task=`83bb4ac7-e4cb-4b57-848c-159f3af5e302`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:31:11,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:11,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132011 s, 77.6 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:31:11,802::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:31:11,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`2acc075e-4634-4a72-b128-524e33b05fe9`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:11,839::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:11,845::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:11,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:11,848::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:31:11,852::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:11,852::task::1191::Storage.TaskManager.Task::(prepare) Task=`2acc075e-4634-4a72-b128-524e33b05fe9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:11,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`2acc075e-4634-4a72-b128-524e33b05fe9`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:11,853::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:11,853::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:11,853::task::993::Storage.TaskManager.Task::(_decref) Task=`2acc075e-4634-4a72-b128-524e33b05fe9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:31:11,865::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:31:11,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`19cb202e-a861-43de-84ba-a2341cdee97e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:31:11,866::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:31:11,866::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6fd2ebf6-21f6-4fd9-8da2-d16434a7ae14`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:31:11,867::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:31:11,867::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:31:11,867::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6fd2ebf6-21f6-4fd9-8da2-d16434a7ae14`::Granted request >Thread-34::DEBUG::2014-08-11 14:31:11,868::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`19cb202e-a861-43de-84ba-a2341cdee97e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:31:11,868::task::993::Storage.TaskManager.Task::(_decref) Task=`19cb202e-a861-43de-84ba-a2341cdee97e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:31:11,888::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082508288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:11,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`19cb202e-a861-43de-84ba-a2341cdee97e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082508288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:11,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`19cb202e-a861-43de-84ba-a2341cdee97e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:31:11,889::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:31:11,889::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:31:11,890::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:31:11,890::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:31:11,890::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:31:11,890::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:31:11,891::task::993::Storage.TaskManager.Task::(_decref) Task=`19cb202e-a861-43de-84ba-a2341cdee97e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:31:13,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:13,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01313 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:15,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:15,626::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131797 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:16,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:16,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:16,940::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:31:16,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f9aa8f6-005f-4113-a892-fa4450242fd8`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:16,995::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:31:16,996::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423841', 'lastCheck': '9.5', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:31:16,996::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f9aa8f6-005f-4113-a892-fa4450242fd8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423841', 'lastCheck': '9.5', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:31:16,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f9aa8f6-005f-4113-a892-fa4450242fd8`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:16,996::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:16,997::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:16,997::task::993::Storage.TaskManager.Task::(_decref) Task=`1f9aa8f6-005f-4113-a892-fa4450242fd8`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:31:17,018::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:31:17,517::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:17,518::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:17,518::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:17,518::__init__::232::IOProcess::(_processLogs) (5184) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:17,519::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|(5184) Queuing response >Thread-16::DEBUG::2014-08-11 14:31:17,520::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:31:17,539::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425142 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:31:17,540::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:17,541::__init__::232::IOProcess::(_processLogs) (5175) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:17,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:17,541::__init__::232::IOProcess::(_processLogs) (5185) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:17,542::__init__::232::IOProcess::(_processLogs) (5185) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:17,543::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:17,544::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:17,545::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:17,545::__init__::232::IOProcess::(_processLogs) (5186) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:17,545::__init__::232::IOProcess::(_processLogs) (5186) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:17,545::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:17,546::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:17,546::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:17,546::__init__::232::IOProcess::(_processLogs) (5187) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:17,547::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|(5187) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:17,548::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:17,548::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:17,549::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:17,550::__init__::232::IOProcess::(_processLogs) (5188) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:17,550::__init__::232::IOProcess::(_processLogs) (5188) Queuing response >Dummy-33::DEBUG::2014-08-11 14:31:17,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:17,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131741 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:19,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:19,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132593 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:21,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:21,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132673 s, 77.2 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:31:21,903::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-92::DEBUG::2014-08-11 14:31:21,904::task::595::Storage.TaskManager.Task::(_updateState) Task=`8728d8d9-ff7a-41bf-befb-b99c70e3350a`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:21,939::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:21,948::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:21,949::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:21,950::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:31:21,952::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:21,953::task::1191::Storage.TaskManager.Task::(prepare) Task=`8728d8d9-ff7a-41bf-befb-b99c70e3350a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:21,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`8728d8d9-ff7a-41bf-befb-b99c70e3350a`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:21,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:21,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:21,954::task::993::Storage.TaskManager.Task::(_decref) Task=`8728d8d9-ff7a-41bf-befb-b99c70e3350a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:31:21,965::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 14:31:21,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`15e6d918-b412-4957-8d92-31dcb842cfc0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:31:21,966::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:31:21,966::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`46e524fa-1078-4f43-96ff-e8861c67db28`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:31:21,967::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:31:21,967::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:31:21,967::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`46e524fa-1078-4f43-96ff-e8861c67db28`::Granted request >Thread-34::DEBUG::2014-08-11 14:31:21,968::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`15e6d918-b412-4957-8d92-31dcb842cfc0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:31:21,968::task::993::Storage.TaskManager.Task::(_decref) Task=`15e6d918-b412-4957-8d92-31dcb842cfc0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:31:21,993::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082442752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:21,994::task::1191::Storage.TaskManager.Task::(prepare) Task=`15e6d918-b412-4957-8d92-31dcb842cfc0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082442752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:21,994::task::595::Storage.TaskManager.Task::(_updateState) Task=`15e6d918-b412-4957-8d92-31dcb842cfc0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:31:21,994::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:31:21,994::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:31:21,995::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:31:21,995::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:31:21,995::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:31:21,995::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:31:21,996::task::993::Storage.TaskManager.Task::(_decref) Task=`15e6d918-b412-4957-8d92-31dcb842cfc0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:31:23,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:23,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131098 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:25,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:25,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131572 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:26,955::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:26,956::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:27,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:31:27,550::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:27,551::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:27,551::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:27,551::__init__::232::IOProcess::(_processLogs) (5189) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:27,552::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|(5189) Queuing response >Thread-16::DEBUG::2014-08-11 14:31:27,553::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:31:27,572::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422562 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:31:27,574::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:27,574::__init__::232::IOProcess::(_processLogs) (5185) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:27,574::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:27,574::__init__::232::IOProcess::(_processLogs) (5190) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:27,575::__init__::232::IOProcess::(_processLogs) (5190) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:27,577::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:27,577::__init__::232::IOProcess::(_processLogs) (5191) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:27,578::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:27,578::__init__::232::IOProcess::(_processLogs) (5192) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:27,578::__init__::232::IOProcess::(_processLogs) (5192) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:27,579::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:27,580::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:27,580::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:27,582::__init__::232::IOProcess::(_processLogs) (5193) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:27,582::__init__::232::IOProcess::(_processLogs) (5193) Queuing response >Dummy-33::DEBUG::2014-08-11 14:31:27,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:27,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132763 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:29,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:30,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133859 s, 76.5 MB/s\n'; <rc> = 0 >Thread-92::DEBUG::2014-08-11 14:31:32,010::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:31:32,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`120b4a0f-c5c9-4cf3-98f4-77ba3a94699b`::moving from state init -> state preparing >Dummy-33::DEBUG::2014-08-11 14:31:32,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:32,060::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:31:32,040::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:32,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:32,068::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::INFO::2014-08-11 14:31:32,079::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:32,082::task::1191::Storage.TaskManager.Task::(prepare) Task=`120b4a0f-c5c9-4cf3-98f4-77ba3a94699b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:32,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`120b4a0f-c5c9-4cf3-98f4-77ba3a94699b`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:32,086::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:32,087::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:32,088::task::993::Storage.TaskManager.Task::(_decref) Task=`120b4a0f-c5c9-4cf3-98f4-77ba3a94699b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:31:32,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0376243 s, 27.2 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 14:31:32,134::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:31:32,138::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c514a9e-261c-48cb-a0da-2adac688e8e0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:31:32,140::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:31:32,147::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e77cbd5a-ffd4-4afd-a974-bc981276f718`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:31:32,150::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-92::DEBUG::2014-08-11 14:31:32,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`11601e11-97ee-4bf7-90b4-b2405b245eb2`::moving from state init -> state preparing >Thread-34::DEBUG::2014-08-11 14:31:32,154::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-92::INFO::2014-08-11 14:31:32,155::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-34::DEBUG::2014-08-11 14:31:32,157::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e77cbd5a-ffd4-4afd-a974-bc981276f718`::Granted request >Thread-92::INFO::2014-08-11 14:31:32,159::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422562', 'lastCheck': '4.6', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:31:32,163::task::1191::Storage.TaskManager.Task::(prepare) Task=`11601e11-97ee-4bf7-90b4-b2405b245eb2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422562', 'lastCheck': '4.6', 'valid': True}} >Thread-34::DEBUG::2014-08-11 14:31:32,163::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6c514a9e-261c-48cb-a0da-2adac688e8e0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:31:32,167::task::993::Storage.TaskManager.Task::(_decref) Task=`6c514a9e-261c-48cb-a0da-2adac688e8e0`::ref 1 aborting False >Thread-92::DEBUG::2014-08-11 14:31:32,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`11601e11-97ee-4bf7-90b4-b2405b245eb2`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:32,178::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:32,184::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:32,190::task::993::Storage.TaskManager.Task::(_decref) Task=`11601e11-97ee-4bf7-90b4-b2405b245eb2`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:31:32,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:31:32,320::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082442752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:32,353::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c514a9e-261c-48cb-a0da-2adac688e8e0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082442752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:32,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c514a9e-261c-48cb-a0da-2adac688e8e0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:31:32,372::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:31:32,374::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:31:32,381::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:31:32,387::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:31:32,389::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:31:32,392::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:31:32,396::task::993::Storage.TaskManager.Task::(_decref) Task=`6c514a9e-261c-48cb-a0da-2adac688e8e0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:31:34,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:34,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0190792 s, 53.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:36,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:36,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131597 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:37,076::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:37,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:37,140::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:31:37,582::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:37,583::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:37,583::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:37,583::__init__::232::IOProcess::(_processLogs) (5194) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:37,583::__init__::232::IOProcess::(_processLogs) (5194) Queuing response >Thread-16::DEBUG::2014-08-11 14:31:37,584::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:31:37,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414328 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:31:37,604::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:37,605::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:37,605::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:37,605::__init__::232::IOProcess::(_processLogs) (5195) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:37,606::__init__::232::IOProcess::(_processLogs) (5195) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:37,607::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:37,607::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:37,608::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:37,608::__init__::232::IOProcess::(_processLogs) (5196) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:37,609::__init__::232::IOProcess::(_processLogs) (5196) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:37,609::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:37,609::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:37,610::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:37,610::__init__::232::IOProcess::(_processLogs) (5197) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:37,611::__init__::232::IOProcess::(_processLogs) (5197) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:37,611::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:37,612::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:37,612::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:37,613::__init__::232::IOProcess::(_processLogs) (5198) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:37,613::__init__::232::IOProcess::(_processLogs) (5198) Queuing response >Dummy-33::DEBUG::2014-08-11 14:31:38,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:38,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131571 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:40,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:40,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132223 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:42,147::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:42,149::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:42,240::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:31:42,418::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:31:42,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`e338606e-1b87-489e-b1bf-75a26c432ec2`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:42,419::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-92::INFO::2014-08-11 14:31:42,432::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:42,432::task::1191::Storage.TaskManager.Task::(prepare) Task=`e338606e-1b87-489e-b1bf-75a26c432ec2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:42,432::task::595::Storage.TaskManager.Task::(_updateState) Task=`e338606e-1b87-489e-b1bf-75a26c432ec2`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:42,432::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:42,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:42,433::task::993::Storage.TaskManager.Task::(_decref) Task=`e338606e-1b87-489e-b1bf-75a26c432ec2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:31:42,445::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:31:42,446::task::595::Storage.TaskManager.Task::(_updateState) Task=`78a1e576-41f4-4f40-aa75-0e962045d40a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:31:42,446::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:31:42,447::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`52f21eea-de9a-4451-98ad-2c5e004678a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:31:42,447::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:31:42,447::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:31:42,448::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`52f21eea-de9a-4451-98ad-2c5e004678a8`::Granted request >Thread-34::DEBUG::2014-08-11 14:31:42,448::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`78a1e576-41f4-4f40-aa75-0e962045d40a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:31:42,449::task::993::Storage.TaskManager.Task::(_decref) Task=`78a1e576-41f4-4f40-aa75-0e962045d40a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:31:42,468::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082442752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:42,469::task::1191::Storage.TaskManager.Task::(prepare) Task=`78a1e576-41f4-4f40-aa75-0e962045d40a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863082442752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:42,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`78a1e576-41f4-4f40-aa75-0e962045d40a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:31:42,469::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:31:42,470::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:31:42,470::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:31:42,470::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:31:42,470::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:31:42,471::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:31:42,471::task::993::Storage.TaskManager.Task::(_decref) Task=`78a1e576-41f4-4f40-aa75-0e962045d40a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:31:42,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:42,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208057 s, 49.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:44,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:44,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131309 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:47,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:47,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203289 s, 50.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:47,247::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:47,249::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:47,340::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:31:47,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b2b1198-3c31-4c85-a12a-46c0e6399164`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:47,451::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-92::INFO::2014-08-11 14:31:47,452::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414328', 'lastCheck': '9.8', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:31:47,452::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b2b1198-3c31-4c85-a12a-46c0e6399164`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414328', 'lastCheck': '9.8', 'valid': True}} >Thread-92::DEBUG::2014-08-11 14:31:47,452::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b2b1198-3c31-4c85-a12a-46c0e6399164`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:47,452::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:47,452::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:47,453::task::993::Storage.TaskManager.Task::(_decref) Task=`4b2b1198-3c31-4c85-a12a-46c0e6399164`::ref 0 aborting False >Thread-92::DEBUG::2014-08-11 14:31:47,473::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:31:47,614::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:47,614::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:47,615::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:47,615::__init__::232::IOProcess::(_processLogs) (5199) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:47,616::__init__::232::IOProcess::(_processLogs) DEBUG|(5199) Queuing response >Thread-16::DEBUG::2014-08-11 14:31:47,617::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:31:47,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419972 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:31:47,637::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:47,637::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:47,637::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:47,638::__init__::232::IOProcess::(_processLogs) (5200) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:47,638::__init__::232::IOProcess::(_processLogs) DEBUG|(5200) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:47,639::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:47,639::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:47,640::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:47,640::__init__::232::IOProcess::(_processLogs) DEBUG|(5201) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:47,640::__init__::232::IOProcess::(_processLogs) (5201) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:47,641::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:47,642::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:47,642::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:47,642::__init__::232::IOProcess::(_processLogs) (5202) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:47,642::__init__::232::IOProcess::(_processLogs) (5202) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:47,643::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:47,644::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:47,644::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:47,644::__init__::232::IOProcess::(_processLogs) (5203) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:47,644::__init__::232::IOProcess::(_processLogs) (5203) Queuing response >Dummy-33::DEBUG::2014-08-11 14:31:49,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:49,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132158 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:51,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:51,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134202 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:52,345::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:52,346::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:52,347::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:31:52,490::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:31:52,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`4713f7f5-0bc8-46b0-a6dd-7217054dd73e`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:31:52,491::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-92::INFO::2014-08-11 14:31:52,504::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:52,505::task::1191::Storage.TaskManager.Task::(prepare) Task=`4713f7f5-0bc8-46b0-a6dd-7217054dd73e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:31:52,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`4713f7f5-0bc8-46b0-a6dd-7217054dd73e`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:31:52,505::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:31:52,505::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:31:52,505::task::993::Storage.TaskManager.Task::(_decref) Task=`4713f7f5-0bc8-46b0-a6dd-7217054dd73e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:31:52,517::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:31:52,518::task::595::Storage.TaskManager.Task::(_updateState) Task=`18383ed6-c8ad-41c9-b950-5d83a5b76479`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:31:52,518::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:31:52,519::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f527913-9bbd-475c-b1cf-87686510bee4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:31:52,519::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:31:52,520::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:31:52,520::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f527913-9bbd-475c-b1cf-87686510bee4`::Granted request >Thread-34::DEBUG::2014-08-11 14:31:52,521::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`18383ed6-c8ad-41c9-b950-5d83a5b76479`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:31:52,521::task::993::Storage.TaskManager.Task::(_decref) Task=`18383ed6-c8ad-41c9-b950-5d83a5b76479`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:31:52,541::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:52,541::task::1191::Storage.TaskManager.Task::(prepare) Task=`18383ed6-c8ad-41c9-b950-5d83a5b76479`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:31:52,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`18383ed6-c8ad-41c9-b950-5d83a5b76479`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:31:52,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:31:52,542::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:31:52,542::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:31:52,543::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:31:52,543::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:31:52,543::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:31:52,543::task::993::Storage.TaskManager.Task::(_decref) Task=`18383ed6-c8ad-41c9-b950-5d83a5b76479`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:31:53,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:53,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139075 s, 73.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:55,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:55,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132533 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:31:57,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:57,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132328 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:57,356::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:57,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:31:57,440::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:31:57,646::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:57,647::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:57,647::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:57,647::__init__::232::IOProcess::(_processLogs) (5204) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:57,648::__init__::232::IOProcess::(_processLogs) (5204) Queuing response >Thread-16::DEBUG::2014-08-11 14:31:57,648::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:31:57,666::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419574 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:31:57,667::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:57,667::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:57,668::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:57,668::__init__::232::IOProcess::(_processLogs) (5205) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:31:57,669::__init__::232::IOProcess::(_processLogs) DEBUG|(5205) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:57,670::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:57,671::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:57,671::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:57,671::__init__::232::IOProcess::(_processLogs) (5206) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:57,672::__init__::232::IOProcess::(_processLogs) DEBUG|(5206) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:57,673::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:57,673::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:57,673::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:57,673::__init__::232::IOProcess::(_processLogs) (5207) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:57,674::__init__::232::IOProcess::(_processLogs) DEBUG|(5207) Queuing response >Thread-14::DEBUG::2014-08-11 14:31:57,675::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:31:57,675::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:31:57,676::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:31:57,676::__init__::232::IOProcess::(_processLogs) (5208) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:31:57,676::__init__::232::IOProcess::(_processLogs) DEBUG|(5208) Queuing response >Dummy-33::DEBUG::2014-08-11 14:31:59,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:31:59,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131951 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:01,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:01,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133727 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:02,449::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:02,456::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:02,544::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-92::DEBUG::2014-08-11 14:32:02,563::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-92::DEBUG::2014-08-11 14:32:02,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`d10bdb1f-a4e9-4f2c-a4ef-598b7fcf4468`::moving from state init -> state preparing >Thread-92::INFO::2014-08-11 14:32:02,564::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Detector thread::DEBUG::2014-08-11 14:32:02,577::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:56971 >Detector thread::DEBUG::2014-08-11 14:32:02,578::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:56971 >Detector thread::DEBUG::2014-08-11 14:32:02,579::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:56971 >Detector thread::DEBUG::2014-08-11 14:32:02,580::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 56971) >Thread-92::INFO::2014-08-11 14:32:02,582::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:32:02,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`d10bdb1f-a4e9-4f2c-a4ef-598b7fcf4468`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-92::DEBUG::2014-08-11 14:32:02,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`d10bdb1f-a4e9-4f2c-a4ef-598b7fcf4468`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-08-11 14:32:02,588::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-08-11 14:32:02,589::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-08-11 14:32:02,591::task::993::Storage.TaskManager.Task::(_decref) Task=`d10bdb1f-a4e9-4f2c-a4ef-598b7fcf4468`::ref 0 aborting False >Thread-93::DEBUG::2014-08-11 14:32:02,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4e2e85a-4667-4faf-922d-3c326003d22a`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:32:02,596::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-93::INFO::2014-08-11 14:32:02,597::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419574', 'lastCheck': '4.9', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:32:02,599::task::1191::Storage.TaskManager.Task::(prepare) Task=`c4e2e85a-4667-4faf-922d-3c326003d22a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419574', 'lastCheck': '4.9', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:32:02,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4e2e85a-4667-4faf-922d-3c326003d22a`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:02,601::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:32:02,602::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:32:02,603::task::993::Storage.TaskManager.Task::(_decref) Task=`c4e2e85a-4667-4faf-922d-3c326003d22a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:32:02,609::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:32:02,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`02f22b15-2a4f-4a9b-868e-cdb0c39a7973`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:32:02,614::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:32:02,618::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`60643e17-cb0e-41ae-8d8c-d5908511fee0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:32:02,619::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:32:02,622::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:32:02,623::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`60643e17-cb0e-41ae-8d8c-d5908511fee0`::Granted request >Thread-34::DEBUG::2014-08-11 14:32:02,627::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`02f22b15-2a4f-4a9b-868e-cdb0c39a7973`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:32:02,628::task::993::Storage.TaskManager.Task::(_decref) Task=`02f22b15-2a4f-4a9b-868e-cdb0c39a7973`::ref 1 aborting False >Thread-93::DEBUG::2014-08-11 14:32:02,677::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:32:02,685::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:02,688::task::1191::Storage.TaskManager.Task::(prepare) Task=`02f22b15-2a4f-4a9b-868e-cdb0c39a7973`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:02,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`02f22b15-2a4f-4a9b-868e-cdb0c39a7973`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:32:02,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:32:02,696::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:32:02,702::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:32:02,706::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:32:02,709::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:32:02,712::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:32:02,720::task::993::Storage.TaskManager.Task::(_decref) Task=`02f22b15-2a4f-4a9b-868e-cdb0c39a7973`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:32:03,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:03,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132975 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:05,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:05,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132782 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:07,549::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:07,551::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:32:07,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:07,660::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:32:07,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134013 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:07,679::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:07,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:07,681::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:07,682::__init__::232::IOProcess::(_processLogs) (5209) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:32:07,684::__init__::232::IOProcess::(_processLogs) DEBUG|(5209) Queuing response >Thread-16::DEBUG::2014-08-11 14:32:07,686::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:32:07,713::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00705089 s, 64.5 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:07,716::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:07,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:07,718::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:07,719::__init__::232::IOProcess::(_processLogs) (5210) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:32:07,721::__init__::232::IOProcess::(_processLogs) DEBUG|(5210) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:07,723::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:07,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:07,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:07,727::__init__::232::IOProcess::(_processLogs) (5211) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:07,728::__init__::232::IOProcess::(_processLogs) DEBUG|(5211) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:07,730::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:07,731::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:07,733::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:07,734::__init__::232::IOProcess::(_processLogs) (5212) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:07,735::__init__::232::IOProcess::(_processLogs) (5212) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:07,737::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:07,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:07,739::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:07,740::__init__::232::IOProcess::(_processLogs) (5213) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:07,741::__init__::232::IOProcess::(_processLogs) DEBUG|(5213) Queuing response >Dummy-33::DEBUG::2014-08-11 14:32:09,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:09,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131836 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:32:11,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`07690172-1fa5-43f4-a9cf-794038cc25c3`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:32:11,016::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:32:11,017::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:11,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:11,018::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:11,018::__init__::232::IOProcess::(_processLogs) (5214) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:32:11,019::__init__::232::IOProcess::(_processLogs) DEBUG|(5214) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:11,021::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:11,021::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:11,021::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:11,021::__init__::232::IOProcess::(_processLogs) (5215) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:32:11,022::__init__::232::IOProcess::(_processLogs) DEBUG|(5215) Queuing response >Thread-55::INFO::2014-08-11 14:32:11,022::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:32:11,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`07690172-1fa5-43f4-a9cf-794038cc25c3`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:32:11,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`07690172-1fa5-43f4-a9cf-794038cc25c3`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:32:11,024::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:32:11,024::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:32:11,024::task::993::Storage.TaskManager.Task::(_decref) Task=`07690172-1fa5-43f4-a9cf-794038cc25c3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:32:11,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:11,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134169 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:12,664::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:12,665::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:32:12,739::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::DEBUG::2014-08-11 14:32:12,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`631f923d-47e5-4615-8329-e7a861cc6e5a`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:32:12,744::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:12,746::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::INFO::2014-08-11 14:32:12,759::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:12,759::task::1191::Storage.TaskManager.Task::(prepare) Task=`631f923d-47e5-4615-8329-e7a861cc6e5a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:12,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`631f923d-47e5-4615-8329-e7a861cc6e5a`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:12,760::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:32:12,760::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:32:12,760::task::993::Storage.TaskManager.Task::(_decref) Task=`631f923d-47e5-4615-8329-e7a861cc6e5a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:32:12,771::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:32:12,772::task::595::Storage.TaskManager.Task::(_updateState) Task=`54610b9b-63b2-4d31-8cc4-7750509b05af`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:32:12,772::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:32:12,775::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`01bebcf1-e697-4e16-808a-57c38ae9fd6e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:32:12,775::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:32:12,775::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:32:12,775::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`01bebcf1-e697-4e16-808a-57c38ae9fd6e`::Granted request >Thread-34::DEBUG::2014-08-11 14:32:12,776::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`54610b9b-63b2-4d31-8cc4-7750509b05af`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:32:12,776::task::993::Storage.TaskManager.Task::(_decref) Task=`54610b9b-63b2-4d31-8cc4-7750509b05af`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:32:12,796::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:12,797::task::1191::Storage.TaskManager.Task::(prepare) Task=`54610b9b-63b2-4d31-8cc4-7750509b05af`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:12,797::task::595::Storage.TaskManager.Task::(_updateState) Task=`54610b9b-63b2-4d31-8cc4-7750509b05af`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:32:12,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:32:12,797::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:32:12,798::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:32:12,798::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:32:12,798::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:32:12,798::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:32:12,799::task::993::Storage.TaskManager.Task::(_decref) Task=`54610b9b-63b2-4d31-8cc4-7750509b05af`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:32:13,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:13,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132456 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:15,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:15,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131812 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:17,743::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:17,744::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:32:17,745::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:17,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:32:17,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`312a97a1-f9d6-4385-89f4-a902820d0f74`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:32:17,841::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-93::INFO::2014-08-11 14:32:17,861::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-14::DEBUG::2014-08-11 14:32:17,863::__init__::232::IOProcess::(_processLogs) (5216) Got request for method 'statvfs' >Thread-93::INFO::2014-08-11 14:32:17,865::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00705089', 'lastCheck': '10.1', 'valid': True}} >Thread-16::DEBUG::2014-08-11 14:32:17,866::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00085771 s, 530 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:17,867::__init__::232::IOProcess::(_processLogs) (5216) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:17,869::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:32:17,869::task::1191::Storage.TaskManager.Task::(prepare) Task=`312a97a1-f9d6-4385-89f4-a902820d0f74`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00705089', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 14:32:17,871::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:17,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-93::DEBUG::2014-08-11 14:32:17,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`312a97a1-f9d6-4385-89f4-a902820d0f74`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:17,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:32:17,875::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-93::DEBUG::2014-08-11 14:32:17,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:32:17,877::__init__::232::IOProcess::(_processLogs) (5217) Got request for method 'statvfs' >Thread-93::DEBUG::2014-08-11 14:32:17,878::task::993::Storage.TaskManager.Task::(_decref) Task=`312a97a1-f9d6-4385-89f4-a902820d0f74`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:17,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:32:17,880::__init__::232::IOProcess::(_processLogs) DEBUG|(5217) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:17,887::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:17,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:17,890::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:17,892::__init__::232::IOProcess::(_processLogs) (5218) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:17,893::__init__::232::IOProcess::(_processLogs) (5218) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:17,895::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:17,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:17,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:17,899::__init__::232::IOProcess::(_processLogs) (5219) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:17,901::__init__::232::IOProcess::(_processLogs) DEBUG|(5219) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:17,903::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:17,905::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:17,906::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:17,908::__init__::232::IOProcess::(_processLogs) (5220) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:17,909::__init__::232::IOProcess::(_processLogs) (5220) Queuing response >Thread-93::DEBUG::2014-08-11 14:32:17,926::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:32:17,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:18,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132884 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:20,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:20,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134831 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:22,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:22,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132113 s, 77.5 MB/s\n'; <rc> = 0 >Thread-93::DEBUG::2014-08-11 14:32:22,814::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::DEBUG::2014-08-11 14:32:22,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`5370023f-42dd-4692-ad41-cde5f8c7f872`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:32:22,842::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-93::INFO::2014-08-11 14:32:22,855::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:22,855::task::1191::Storage.TaskManager.Task::(prepare) Task=`5370023f-42dd-4692-ad41-cde5f8c7f872`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:22,856::task::595::Storage.TaskManager.Task::(_updateState) Task=`5370023f-42dd-4692-ad41-cde5f8c7f872`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:22,856::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:32:22,856::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:32:22,856::task::993::Storage.TaskManager.Task::(_decref) Task=`5370023f-42dd-4692-ad41-cde5f8c7f872`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:32:22,871::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:32:22,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`f65d907d-3cfc-4fed-9cb8-15d748b84a8f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:32:22,872::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:32:22,873::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`845b2f2e-32ce-485b-a061-ca0b73623941`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:32:22,873::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:32:22,873::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:32:22,873::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`845b2f2e-32ce-485b-a061-ca0b73623941`::Granted request >Thread-34::DEBUG::2014-08-11 14:32:22,874::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f65d907d-3cfc-4fed-9cb8-15d748b84a8f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:32:22,874::task::993::Storage.TaskManager.Task::(_decref) Task=`f65d907d-3cfc-4fed-9cb8-15d748b84a8f`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:22,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:22,896::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:22,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:32:22,898::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:22,898::task::1191::Storage.TaskManager.Task::(prepare) Task=`f65d907d-3cfc-4fed-9cb8-15d748b84a8f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079231488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:22,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`f65d907d-3cfc-4fed-9cb8-15d748b84a8f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:32:22,899::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:32:22,899::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:32:22,899::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:32:22,900::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:32:22,900::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:32:22,900::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:32:22,900::task::993::Storage.TaskManager.Task::(_decref) Task=`f65d907d-3cfc-4fed-9cb8-15d748b84a8f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:32:24,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:24,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132194 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:26,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:26,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132642 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:27,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:27,902::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:32:27,905::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:32:27,946::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:27,947::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:32:27,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:27,950::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:27,956::__init__::232::IOProcess::(_processLogs) (5221) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:32:27,957::__init__::232::IOProcess::(_processLogs) (5221) Queuing response >Thread-16::DEBUG::2014-08-11 14:32:27,963::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:32:27,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:27,964::__init__::232::IOProcess::(_processLogs) (5222) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 14:32:27,965::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:32:27,966::__init__::232::IOProcess::(_processLogs) (5223) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 14:32:27,967::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:32:27,969::__init__::232::IOProcess::(_processLogs) (5224) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:27,970::__init__::232::IOProcess::(_processLogs) (5225) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:32:27,970::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 14:32:27,970::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:32:27,971::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:32:27,971::__init__::232::IOProcess::(_processLogs) (5226) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:32:27,972::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:32:27,990::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427314 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:27,992::__init__::232::IOProcess::(_processLogs) (5227) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:32:27,993::__init__::232::IOProcess::(_processLogs) (5228) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:27,994::__init__::232::IOProcess::(_processLogs) (5229) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:27,995::__init__::232::IOProcess::(_processLogs) (5230) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:32:28,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:28,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133456 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:30,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:30,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132855 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:32,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:32,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272816 s, 37.5 MB/s\n'; <rc> = 0 >Thread-93::DEBUG::2014-08-11 14:32:32,915::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::DEBUG::2014-08-11 14:32:32,918::task::595::Storage.TaskManager.Task::(_updateState) Task=`a48977a5-ebc6-48dc-9aa2-71d0e264ac56`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:32:32,941::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-93::INFO::2014-08-11 14:32:32,958::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:32,960::task::1191::Storage.TaskManager.Task::(prepare) Task=`a48977a5-ebc6-48dc-9aa2-71d0e264ac56`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:32,960::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:32:32,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`a48977a5-ebc6-48dc-9aa2-71d0e264ac56`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:32,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:32:32,963::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:32,965::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:32:32,965::task::993::Storage.TaskManager.Task::(_decref) Task=`a48977a5-ebc6-48dc-9aa2-71d0e264ac56`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:32,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:32:32,978::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:32:32,979::task::595::Storage.TaskManager.Task::(_updateState) Task=`987f4a81-82c4-4621-ad92-0b2d6d130155`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:32:32,979::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:32:32,980::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`77c0f9e6-6cca-4789-8ce4-3fbcd0a59987`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:32:32,980::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:32:32,980::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:32:32,981::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`77c0f9e6-6cca-4789-8ce4-3fbcd0a59987`::Granted request >Thread-34::DEBUG::2014-08-11 14:32:32,981::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`987f4a81-82c4-4621-ad92-0b2d6d130155`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:32:32,981::task::993::Storage.TaskManager.Task::(_decref) Task=`987f4a81-82c4-4621-ad92-0b2d6d130155`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:32:33,001::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079165952', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:33,002::task::1191::Storage.TaskManager.Task::(prepare) Task=`987f4a81-82c4-4621-ad92-0b2d6d130155`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079165952', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:33,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`987f4a81-82c4-4621-ad92-0b2d6d130155`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:32:33,002::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:32:33,002::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:32:33,003::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:32:33,003::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:32:33,003::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:32:33,003::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:32:33,004::task::993::Storage.TaskManager.Task::(_decref) Task=`987f4a81-82c4-4621-ad92-0b2d6d130155`::ref 0 aborting False >Thread-93::DEBUG::2014-08-11 14:32:33,022::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f72ba11-4533-4000-a64a-c9d5846faaaf`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:32:33,023::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-93::INFO::2014-08-11 14:32:33,023::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427314', 'lastCheck': '5.0', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:32:33,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f72ba11-4533-4000-a64a-c9d5846faaaf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427314', 'lastCheck': '5.0', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:32:33,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f72ba11-4533-4000-a64a-c9d5846faaaf`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:33,024::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:32:33,024::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:32:33,024::task::993::Storage.TaskManager.Task::(_decref) Task=`2f72ba11-4533-4000-a64a-c9d5846faaaf`::ref 0 aborting False >Thread-93::DEBUG::2014-08-11 14:32:33,044::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:32:34,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:34,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013239 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:36,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:36,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013135 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:37,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:37,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:32:37,997::__init__::232::IOProcess::(_processLogs) (5231) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:38,044::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:32:38,046::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:32:38,065::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417786 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:38,066::__init__::232::IOProcess::(_processLogs) (5232) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:32:38,068::__init__::232::IOProcess::(_processLogs) (5233) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:38,069::__init__::232::IOProcess::(_processLogs) (5234) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:38,070::__init__::232::IOProcess::(_processLogs) (5235) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:32:38,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:38,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132272 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:40,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:40,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131381 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:42,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:42,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132584 s, 77.2 MB/s\n'; <rc> = 0 >Thread-93::DEBUG::2014-08-11 14:32:43,022::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::DEBUG::2014-08-11 14:32:43,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7cbeb27-a283-455e-98e0-1ee40402cef4`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:32:43,023::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-93::INFO::2014-08-11 14:32:43,036::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:43,036::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7cbeb27-a283-455e-98e0-1ee40402cef4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:43,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7cbeb27-a283-455e-98e0-1ee40402cef4`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:43,037::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:32:43,037::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:32:43,037::task::993::Storage.TaskManager.Task::(_decref) Task=`b7cbeb27-a283-455e-98e0-1ee40402cef4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:32:43,047::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:32:43,048::task::595::Storage.TaskManager.Task::(_updateState) Task=`44571bd8-d3b9-4fd1-9825-0e0b23443f41`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:32:43,048::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:32:43,049::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5b7c0fe6-055b-4844-aca9-04e270b03e63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:32:43,049::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:32:43,050::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:32:43,050::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5b7c0fe6-055b-4844-aca9-04e270b03e63`::Granted request >Thread-34::DEBUG::2014-08-11 14:32:43,053::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`44571bd8-d3b9-4fd1-9825-0e0b23443f41`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:43,053::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:32:43,054::task::993::Storage.TaskManager.Task::(_decref) Task=`44571bd8-d3b9-4fd1-9825-0e0b23443f41`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:43,060::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:43,066::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:32:43,091::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079165952', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:43,092::task::1191::Storage.TaskManager.Task::(prepare) Task=`44571bd8-d3b9-4fd1-9825-0e0b23443f41`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079165952', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:43,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`44571bd8-d3b9-4fd1-9825-0e0b23443f41`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:32:43,092::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:32:43,092::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:32:43,093::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:32:43,093::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:32:43,093::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:32:43,093::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:32:43,094::task::993::Storage.TaskManager.Task::(_decref) Task=`44571bd8-d3b9-4fd1-9825-0e0b23443f41`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:32:44,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:44,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133435 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:46,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:46,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132211 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:48,072::__init__::232::IOProcess::(_processLogs) (5236) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:32:48,073::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:48,076::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:32:48,161::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca94160d-9701-42c3-856d-a8e67b9aad5c`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:32:48,162::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:48,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:32:48,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00285888 s, 159 kB/s\n'; <rc> = 0 >Thread-93::INFO::2014-08-11 14:32:48,167::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417786', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 14:32:48,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:48,170::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:32:48,171::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:48,173::__init__::232::IOProcess::(_processLogs) (5237) Got request for method 'statvfs' >Thread-93::DEBUG::2014-08-11 14:32:48,171::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca94160d-9701-42c3-856d-a8e67b9aad5c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417786', 'lastCheck': '10.1', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:32:48,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca94160d-9701-42c3-856d-a8e67b9aad5c`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:32:48,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-93::DEBUG::2014-08-11 14:32:48,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:32:48,178::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-93::DEBUG::2014-08-11 14:32:48,178::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:32:48,180::task::993::Storage.TaskManager.Task::(_decref) Task=`ca94160d-9701-42c3-856d-a8e67b9aad5c`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:32:48,180::__init__::232::IOProcess::(_processLogs) (5238) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:48,182::__init__::232::IOProcess::(_processLogs) (5238) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:48,185::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:48,187::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:48,188::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:48,190::__init__::232::IOProcess::(_processLogs) (5239) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:48,192::__init__::232::IOProcess::(_processLogs) DEBUG|(5239) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:48,193::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:48,195::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:48,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:48,198::__init__::232::IOProcess::(_processLogs) (5240) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:48,199::__init__::232::IOProcess::(_processLogs) DEBUG|(5240) Queuing response >Thread-93::DEBUG::2014-08-11 14:32:48,218::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:32:49,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:49,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133004 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:51,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:51,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132556 s, 77.3 MB/s\n'; <rc> = 0 >Thread-93::DEBUG::2014-08-11 14:32:53,105::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::DEBUG::2014-08-11 14:32:53,106::task::595::Storage.TaskManager.Task::(_updateState) Task=`364a6932-87a4-4896-ae38-e54e24c3872a`::moving from state init -> state preparing >Dummy-33::DEBUG::2014-08-11 14:32:53,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-93::INFO::2014-08-11 14:32:53,142::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 14:32:53,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148169 s, 69.1 MB/s\n'; <rc> = 0 >Thread-93::INFO::2014-08-11 14:32:53,185::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:53,185::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:32:53,187::task::1191::Storage.TaskManager.Task::(prepare) Task=`364a6932-87a4-4896-ae38-e54e24c3872a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:32:53,188::task::595::Storage.TaskManager.Task::(_updateState) Task=`364a6932-87a4-4896-ae38-e54e24c3872a`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:32:53,189::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:32:53,190::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:32:53,192::task::993::Storage.TaskManager.Task::(_decref) Task=`364a6932-87a4-4896-ae38-e54e24c3872a`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:53,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:53,201::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:32:53,207::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:32:53,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`c98d7c46-566f-46da-a8ad-f3f91e2c4bc0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:32:53,210::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:32:53,213::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8383d552-027d-466b-b215-ab6016d2d8ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:32:53,214::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:32:53,216::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:32:53,217::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8383d552-027d-466b-b215-ab6016d2d8ea`::Granted request >Thread-34::DEBUG::2014-08-11 14:32:53,220::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c98d7c46-566f-46da-a8ad-f3f91e2c4bc0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:32:53,222::task::993::Storage.TaskManager.Task::(_decref) Task=`c98d7c46-566f-46da-a8ad-f3f91e2c4bc0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:32:53,269::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:53,269::task::1191::Storage.TaskManager.Task::(prepare) Task=`c98d7c46-566f-46da-a8ad-f3f91e2c4bc0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:32:53,269::task::595::Storage.TaskManager.Task::(_updateState) Task=`c98d7c46-566f-46da-a8ad-f3f91e2c4bc0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:32:53,270::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:32:53,270::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:32:53,270::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:32:53,270::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:32:53,271::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:32:53,271::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:32:53,271::task::993::Storage.TaskManager.Task::(_decref) Task=`c98d7c46-566f-46da-a8ad-f3f91e2c4bc0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:32:55,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:55,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132123 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:32:57,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:57,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132164 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:32:58,202::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:58,202::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:58,207::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:32:58,244::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:32:58,243::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:58,246::__init__::232::IOProcess::(_processLogs) (5241) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:58,265::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:32:58,266::__init__::232::IOProcess::(_processLogs) (5241) Queuing response >Thread-16::DEBUG::2014-08-11 14:32:58,269::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000571072 s, 797 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:32:58,271::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:32:58,272::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:58,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:58,275::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:58,275::__init__::232::IOProcess::(_processLogs) (5242) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:32:58,276::__init__::232::IOProcess::(_processLogs) DEBUG|(5242) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:58,277::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:58,277::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:58,278::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:58,278::__init__::232::IOProcess::(_processLogs) (5243) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:58,278::__init__::232::IOProcess::(_processLogs) DEBUG|(5243) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:58,279::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:58,279::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:58,280::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:58,280::__init__::232::IOProcess::(_processLogs) (5244) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:58,281::__init__::232::IOProcess::(_processLogs) DEBUG|(5244) Queuing response >Thread-14::DEBUG::2014-08-11 14:32:58,282::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:32:58,282::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:32:58,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:32:58,282::__init__::232::IOProcess::(_processLogs) (5245) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:32:58,283::__init__::232::IOProcess::(_processLogs) (5245) Queuing response >Dummy-33::DEBUG::2014-08-11 14:32:59,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:32:59,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132661 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:01,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:01,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134128 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:03,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:03,282::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:33:03,284::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Detector thread::DEBUG::2014-08-11 14:33:03,304::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:53598 >Thread-93::DEBUG::2014-08-11 14:33:03,344::task::595::Storage.TaskManager.Task::(_updateState) Task=`985da566-f3d7-4d39-bac1-26f9ab1f7a5f`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:33:03,346::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Detector thread::DEBUG::2014-08-11 14:33:03,347::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:53598 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:03,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 14:33:03,349::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:53598 >Detector thread::DEBUG::2014-08-11 14:33:03,351::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 53598) >Thread-95::DEBUG::2014-08-11 14:33:03,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`fa7bed17-ff6b-4ce9-85c0-c1f2c9c75c0c`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:33:03,356::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:33:03,356::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000571072', 'lastCheck': '5.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:33:03,356::task::1191::Storage.TaskManager.Task::(prepare) Task=`fa7bed17-ff6b-4ce9-85c0-c1f2c9c75c0c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000571072', 'lastCheck': '5.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:33:03,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`fa7bed17-ff6b-4ce9-85c0-c1f2c9c75c0c`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:33:03,357::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:33:03,357::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:33:03,357::task::993::Storage.TaskManager.Task::(_decref) Task=`fa7bed17-ff6b-4ce9-85c0-c1f2c9c75c0c`::ref 0 aborting False >Thread-93::INFO::2014-08-11 14:33:03,362::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:33:03,363::task::1191::Storage.TaskManager.Task::(prepare) Task=`985da566-f3d7-4d39-bac1-26f9ab1f7a5f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:33:03,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`985da566-f3d7-4d39-bac1-26f9ab1f7a5f`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:33:03,363::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:33:03,363::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:33:03,364::task::993::Storage.TaskManager.Task::(_decref) Task=`985da566-f3d7-4d39-bac1-26f9ab1f7a5f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:33:03,374::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:33:03,374::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbc2dfef-4ca4-4da0-829c-76c57f9a323a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:33:03,375::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:33:03,375::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`595c5b87-4f01-4cb7-acbd-7a77623a3265`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:33:03,376::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:33:03,376::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:33:03,376::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`595c5b87-4f01-4cb7-acbd-7a77623a3265`::Granted request >Thread-34::DEBUG::2014-08-11 14:33:03,377::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cbc2dfef-4ca4-4da0-829c-76c57f9a323a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:33:03,377::task::993::Storage.TaskManager.Task::(_decref) Task=`cbc2dfef-4ca4-4da0-829c-76c57f9a323a`::ref 1 aborting False >Thread-93::DEBUG::2014-08-11 14:33:03,387::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:33:03,414::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:03,414::task::1191::Storage.TaskManager.Task::(prepare) Task=`cbc2dfef-4ca4-4da0-829c-76c57f9a323a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:03,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbc2dfef-4ca4-4da0-829c-76c57f9a323a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:33:03,415::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:33:03,415::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:33:03,415::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:33:03,416::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:33:03,416::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:33:03,416::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:33:03,416::task::993::Storage.TaskManager.Task::(_decref) Task=`cbc2dfef-4ca4-4da0-829c-76c57f9a323a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:33:03,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:03,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131531 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:05,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:05,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131951 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:07,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:07,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132522 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:08,285::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:08,285::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:08,342::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:33:08,343::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:33:08,343::__init__::232::IOProcess::(_processLogs) (5246) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:08,363::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:33:08,364::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413589 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:08,365::__init__::232::IOProcess::(_processLogs) (5246) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:08,368::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:08,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:08,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:08,370::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:08,371::__init__::232::IOProcess::(_processLogs) (5247) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:08,373::__init__::232::IOProcess::(_processLogs) DEBUG|(5247) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:08,376::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:08,377::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:08,377::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:08,378::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:08,379::__init__::232::IOProcess::(_processLogs) (5248) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:08,381::__init__::232::IOProcess::(_processLogs) DEBUG|(5248) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:08,383::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:08,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:08,384::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:08,384::__init__::232::IOProcess::(_processLogs) (5249) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:08,385::__init__::232::IOProcess::(_processLogs) (5249) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:08,385::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:08,385::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:08,386::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:08,386::__init__::232::IOProcess::(_processLogs) DEBUG|(5250) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:08,386::__init__::232::IOProcess::(_processLogs) (5250) Queuing response >Dummy-33::DEBUG::2014-08-11 14:33:09,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:09,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134664 s, 76.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:33:11,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`4df97a6f-f180-400f-9e18-495af1bf58a2`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:33:11,127::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:33:11,128::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:11,128::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:11,128::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:11,129::__init__::232::IOProcess::(_processLogs) (5251) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:33:11,129::__init__::232::IOProcess::(_processLogs) DEBUG|(5251) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:11,130::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:11,130::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:11,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:11,131::__init__::232::IOProcess::(_processLogs) DEBUG|(5252) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:33:11,131::__init__::232::IOProcess::(_processLogs) (5252) Queuing response >Thread-55::INFO::2014-08-11 14:33:11,132::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:33:11,132::task::1191::Storage.TaskManager.Task::(prepare) Task=`4df97a6f-f180-400f-9e18-495af1bf58a2`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:33:11,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`4df97a6f-f180-400f-9e18-495af1bf58a2`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:33:11,132::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:33:11,133::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:33:11,133::task::993::Storage.TaskManager.Task::(_decref) Task=`4df97a6f-f180-400f-9e18-495af1bf58a2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:33:11,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:11,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132367 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:13,388::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:13,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:33:13,428::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::DEBUG::2014-08-11 14:33:13,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`63d776e4-0500-4619-9bad-ede00c554f69`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:13,446::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::INFO::2014-08-11 14:33:13,446::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-93::INFO::2014-08-11 14:33:13,461::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:33:13,461::task::1191::Storage.TaskManager.Task::(prepare) Task=`63d776e4-0500-4619-9bad-ede00c554f69`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:33:13,461::task::595::Storage.TaskManager.Task::(_updateState) Task=`63d776e4-0500-4619-9bad-ede00c554f69`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:33:13,461::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:33:13,462::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:33:13,462::task::993::Storage.TaskManager.Task::(_decref) Task=`63d776e4-0500-4619-9bad-ede00c554f69`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:33:13,472::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:33:13,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`21004087-4039-4447-be60-8f28948155ef`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:33:13,473::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:33:13,474::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4e8fad0d-c468-47d8-a895-1270566a3aaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:33:13,474::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:33:13,475::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:33:13,475::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4e8fad0d-c468-47d8-a895-1270566a3aaf`::Granted request >Thread-34::DEBUG::2014-08-11 14:33:13,475::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`21004087-4039-4447-be60-8f28948155ef`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:33:13,476::task::993::Storage.TaskManager.Task::(_decref) Task=`21004087-4039-4447-be60-8f28948155ef`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:33:13,496::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:13,496::task::1191::Storage.TaskManager.Task::(prepare) Task=`21004087-4039-4447-be60-8f28948155ef`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:13,496::task::595::Storage.TaskManager.Task::(_updateState) Task=`21004087-4039-4447-be60-8f28948155ef`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:33:13,496::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:33:13,497::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:33:13,497::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:33:13,497::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:33:13,497::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:33:13,498::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:33:13,498::task::993::Storage.TaskManager.Task::(_decref) Task=`21004087-4039-4447-be60-8f28948155ef`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:33:13,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:13,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133552 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:15,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:15,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131394 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:17,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:17,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132423 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:18,388::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:18,389::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:18,389::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:18,389::__init__::232::IOProcess::(_processLogs) (5253) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:18,441::__init__::232::IOProcess::(_processLogs) (5253) Queuing response >Thread-16::DEBUG::2014-08-11 14:33:18,442::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:18,461::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:18,463::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:33:18,464::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412692 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:18,467::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:18,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:18,470::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:18,470::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:18,471::__init__::232::IOProcess::(_processLogs) (5254) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:18,473::__init__::232::IOProcess::(_processLogs) DEBUG|(5254) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:18,475::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:18,476::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:18,477::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:18,477::__init__::232::IOProcess::(_processLogs) (5255) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:18,477::__init__::232::IOProcess::(_processLogs) (5255) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:18,477::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:18,478::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:18,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:18,478::__init__::232::IOProcess::(_processLogs) (5256) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:18,479::__init__::232::IOProcess::(_processLogs) DEBUG|(5256) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:18,480::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:18,480::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:18,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:18,482::__init__::232::IOProcess::(_processLogs) (5257) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:18,482::__init__::232::IOProcess::(_processLogs) (5257) Queuing response >Thread-93::DEBUG::2014-08-11 14:33:18,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`5df529c8-7ad6-4a55-bb1a-3d079f9a10d8`::moving from state init -> state preparing >Thread-93::INFO::2014-08-11 14:33:18,491::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-93::INFO::2014-08-11 14:33:18,491::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412692', 'lastCheck': '0.0', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:33:18,492::task::1191::Storage.TaskManager.Task::(prepare) Task=`5df529c8-7ad6-4a55-bb1a-3d079f9a10d8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412692', 'lastCheck': '0.0', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:33:18,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`5df529c8-7ad6-4a55-bb1a-3d079f9a10d8`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:33:18,492::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:33:18,492::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:33:18,492::task::993::Storage.TaskManager.Task::(_decref) Task=`5df529c8-7ad6-4a55-bb1a-3d079f9a10d8`::ref 0 aborting False >Thread-93::DEBUG::2014-08-11 14:33:18,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:33:19,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:19,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132547 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:21,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:22,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131841 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:23,481::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:23,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:33:23,510::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::DEBUG::2014-08-11 14:33:23,544::task::595::Storage.TaskManager.Task::(_updateState) Task=`3121d330-b5d8-451d-a497-70f147f026f8`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:23,546::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::INFO::2014-08-11 14:33:23,546::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-93::INFO::2014-08-11 14:33:23,560::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:33:23,560::task::1191::Storage.TaskManager.Task::(prepare) Task=`3121d330-b5d8-451d-a497-70f147f026f8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-93::DEBUG::2014-08-11 14:33:23,561::task::595::Storage.TaskManager.Task::(_updateState) Task=`3121d330-b5d8-451d-a497-70f147f026f8`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:33:23,561::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:33:23,561::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:33:23,561::task::993::Storage.TaskManager.Task::(_decref) Task=`3121d330-b5d8-451d-a497-70f147f026f8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:33:23,573::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:33:23,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cccf255-eab1-4d61-b638-0d8d4d8e3301`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:33:23,574::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:33:23,574::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`913405aa-fba7-4457-9eff-2fb7cbee8be9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:33:23,575::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:33:23,575::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:33:23,575::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`913405aa-fba7-4457-9eff-2fb7cbee8be9`::Granted request >Thread-34::DEBUG::2014-08-11 14:33:23,576::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4cccf255-eab1-4d61-b638-0d8d4d8e3301`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:33:23,576::task::993::Storage.TaskManager.Task::(_decref) Task=`4cccf255-eab1-4d61-b638-0d8d4d8e3301`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:33:23,596::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:23,596::task::1191::Storage.TaskManager.Task::(prepare) Task=`4cccf255-eab1-4d61-b638-0d8d4d8e3301`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1863079100416', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:23,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`4cccf255-eab1-4d61-b638-0d8d4d8e3301`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:33:23,597::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:33:23,597::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:33:23,597::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:33:23,598::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:33:23,598::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:33:23,598::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:33:23,598::task::993::Storage.TaskManager.Task::(_decref) Task=`4cccf255-eab1-4d61-b638-0d8d4d8e3301`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:33:24,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:24,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132018 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:26,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:26,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132717 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:28,147::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:28,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133246 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:28,483::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:28,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:28,484::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:33:28,485::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:33:28,542::__init__::232::IOProcess::(_processLogs) (5258) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:28,561::__init__::232::IOProcess::(_processLogs) (5258) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:28,564::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:33:28,565::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000922813 s, 493 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:28,569::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:28,569::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:28,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:28,572::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:28,573::__init__::232::IOProcess::(_processLogs) (5259) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:28,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:28,575::__init__::232::IOProcess::(_processLogs) DEBUG|(5259) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:28,579::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:28,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:28,581::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:28,581::__init__::232::IOProcess::(_processLogs) (5260) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:28,581::__init__::232::IOProcess::(_processLogs) (5260) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:28,581::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:28,582::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:28,582::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:28,583::__init__::232::IOProcess::(_processLogs) DEBUG|(5261) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:28,583::__init__::232::IOProcess::(_processLogs) (5261) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:28,584::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:28,584::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:28,585::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:28,586::__init__::232::IOProcess::(_processLogs) (5262) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:28,586::__init__::232::IOProcess::(_processLogs) (5262) Queuing response >Dummy-33::DEBUG::2014-08-11 14:33:30,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:30,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134968 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:32,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:32,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0286394 s, 35.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:33,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:33,588::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::DEBUG::2014-08-11 14:33:33,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`06e4348b-fc77-4334-9887-e4054aba5ca6`::moving from state init -> state preparing >Thread-95::DEBUG::2014-08-11 14:33:33,612::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-93::INFO::2014-08-11 14:33:33,644::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::DEBUG::2014-08-11 14:33:33,645::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fd9c71c-2360-4667-99cb-44923d56e1ea`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:33:33,647::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:33,648::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-93::INFO::2014-08-11 14:33:33,649::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000922813', 'lastCheck': '5.1', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:33:33,650::task::1191::Storage.TaskManager.Task::(prepare) Task=`06e4348b-fc77-4334-9887-e4054aba5ca6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000922813', 'lastCheck': '5.1', 'valid': True}} >Thread-93::DEBUG::2014-08-11 14:33:33,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`06e4348b-fc77-4334-9887-e4054aba5ca6`::moving from state preparing -> state finished >Thread-93::DEBUG::2014-08-11 14:33:33,652::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-93::DEBUG::2014-08-11 14:33:33,652::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-93::DEBUG::2014-08-11 14:33:33,652::task::993::Storage.TaskManager.Task::(_decref) Task=`06e4348b-fc77-4334-9887-e4054aba5ca6`::ref 0 aborting False >Thread-95::INFO::2014-08-11 14:33:33,667::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:33:33,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`0fd9c71c-2360-4667-99cb-44923d56e1ea`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:33:33,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fd9c71c-2360-4667-99cb-44923d56e1ea`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:33:33,667::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:33:33,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:33:33,668::task::993::Storage.TaskManager.Task::(_decref) Task=`0fd9c71c-2360-4667-99cb-44923d56e1ea`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:33:33,677::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:33:33,680::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:33:33,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d73e976-50ca-40ac-9e57-daacd3b320ee`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:33:33,684::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:33:33,687::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3d8ff4b6-6f82-4ae7-94a6-e73ca4a35271`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:33:33,688::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:33:33,689::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:33:33,690::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3d8ff4b6-6f82-4ae7-94a6-e73ca4a35271`::Granted request >Thread-34::DEBUG::2014-08-11 14:33:33,693::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3d73e976-50ca-40ac-9e57-daacd3b320ee`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:33:33,693::task::993::Storage.TaskManager.Task::(_decref) Task=`3d73e976-50ca-40ac-9e57-daacd3b320ee`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:33:33,717::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862949994496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:33,717::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d73e976-50ca-40ac-9e57-daacd3b320ee`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862949994496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:33,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d73e976-50ca-40ac-9e57-daacd3b320ee`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:33:33,718::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:33:33,718::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:33:33,718::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:33:33,719::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:33:33,719::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:33:33,719::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:33:33,719::task::993::Storage.TaskManager.Task::(_decref) Task=`3d73e976-50ca-40ac-9e57-daacd3b320ee`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:33:34,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:34,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01318 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:36,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:36,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134757 s, 76.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:38,587::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:38,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:38,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:38,588::__init__::232::IOProcess::(_processLogs) (5263) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:38,588::__init__::232::IOProcess::(_processLogs) DEBUG|(5263) Queuing response >Thread-16::DEBUG::2014-08-11 14:33:38,589::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:33:38,618::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00693327 s, 65.6 kB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:38,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:33:38,662::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:38,665::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:38,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:38,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:38,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:38,672::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:38,673::__init__::232::IOProcess::(_processLogs) (5264) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:38,676::__init__::232::IOProcess::(_processLogs) (5264) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:38,678::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:38,678::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:38,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:38,679::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:38,681::__init__::232::IOProcess::(_processLogs) (5265) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:38,683::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:38,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:38,687::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:38,689::__init__::232::IOProcess::(_processLogs) (5266) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:33:38,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283505 s, 36.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:38,696::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access'DEBUG|(5266) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:38,698::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:38,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:38,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:38,702::__init__::232::IOProcess::(_processLogs) (5267) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:38,703::__init__::232::IOProcess::(_processLogs) (5267) Queuing response >Dummy-33::DEBUG::2014-08-11 14:33:40,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:40,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132022 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:42,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:42,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132473 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:43,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:43,690::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:43,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:33:43,731::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:33:43,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1e0ad42-224d-4b7f-9d24-5ea8a445af16`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:33:43,743::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:33:43,756::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:33:43,757::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1e0ad42-224d-4b7f-9d24-5ea8a445af16`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:33:43,757::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1e0ad42-224d-4b7f-9d24-5ea8a445af16`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:33:43,757::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:33:43,757::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:33:43,758::task::993::Storage.TaskManager.Task::(_decref) Task=`d1e0ad42-224d-4b7f-9d24-5ea8a445af16`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:33:43,774::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:33:43,774::task::595::Storage.TaskManager.Task::(_updateState) Task=`624c1590-79c5-40a6-b279-ca1bb16cbc71`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:33:43,775::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:33:43,777::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f0399a60-0f6e-4900-8006-5290f3f516fc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:33:43,778::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:33:43,778::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:33:43,778::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f0399a60-0f6e-4900-8006-5290f3f516fc`::Granted request >Thread-34::DEBUG::2014-08-11 14:33:43,779::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`624c1590-79c5-40a6-b279-ca1bb16cbc71`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:33:43,779::task::993::Storage.TaskManager.Task::(_decref) Task=`624c1590-79c5-40a6-b279-ca1bb16cbc71`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:33:43,799::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862661963776', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:43,799::task::1191::Storage.TaskManager.Task::(prepare) Task=`624c1590-79c5-40a6-b279-ca1bb16cbc71`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862661963776', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:43,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`624c1590-79c5-40a6-b279-ca1bb16cbc71`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:33:43,800::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:33:43,800::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:33:43,801::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:33:43,801::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:33:43,801::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:33:43,801::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:33:43,802::task::993::Storage.TaskManager.Task::(_decref) Task=`624c1590-79c5-40a6-b279-ca1bb16cbc71`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:33:44,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:44,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131213 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:46,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:46,920::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132643 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:48,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:48,699::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:48,699::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:48,702::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:48,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:48,703::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:48,703::__init__::232::IOProcess::(_processLogs) (5268) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:48,704::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access'DEBUG|(5268) Queuing response >Thread-16::DEBUG::2014-08-11 14:33:48,705::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:33:48,723::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424711 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:48,725::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:33:48,725::__init__::232::IOProcess::(_processLogs) (5265) Got request for method 'access'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:33:48,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:48,726::__init__::232::IOProcess::(_processLogs) (5269) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:48,727::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:48,727::__init__::232::IOProcess::(_processLogs) (5270) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:48,728::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:48,728::__init__::232::IOProcess::(_processLogs) (5271) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:48,729::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:48,730::__init__::232::IOProcess::(_processLogs) (5272) Queuing response >Thread-95::DEBUG::2014-08-11 14:33:48,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`cde1d6fe-7eef-48db-b496-a07e60c7cec7`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:33:48,795::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:33:48,796::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424711', 'lastCheck': '0.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:33:48,796::task::1191::Storage.TaskManager.Task::(prepare) Task=`cde1d6fe-7eef-48db-b496-a07e60c7cec7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424711', 'lastCheck': '0.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:33:48,797::task::595::Storage.TaskManager.Task::(_updateState) Task=`cde1d6fe-7eef-48db-b496-a07e60c7cec7`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:33:48,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:33:48,797::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:33:48,798::task::993::Storage.TaskManager.Task::(_decref) Task=`cde1d6fe-7eef-48db-b496-a07e60c7cec7`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:33:48,818::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:33:48,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:48,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132777 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:50,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:51,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133735 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:53,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:53,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131578 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:53,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:53,709::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:53,710::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:33:53,830::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-95::DEBUG::2014-08-11 14:33:53,833::task::595::Storage.TaskManager.Task::(_updateState) Task=`e89b6d6d-cccf-4573-98c1-6411bc93cab3`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:33:53,834::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:33:53,849::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:33:53,849::task::1191::Storage.TaskManager.Task::(prepare) Task=`e89b6d6d-cccf-4573-98c1-6411bc93cab3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:33:53,850::task::595::Storage.TaskManager.Task::(_updateState) Task=`e89b6d6d-cccf-4573-98c1-6411bc93cab3`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:33:53,850::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:33:53,850::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:33:53,850::task::993::Storage.TaskManager.Task::(_decref) Task=`e89b6d6d-cccf-4573-98c1-6411bc93cab3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:33:53,868::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 14:33:53,871::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc286eb3-e3b0-4ac9-b6e4-e8564ca2c51f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:33:53,872::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:33:53,873::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fa557618-77a5-4784-b84f-2e7514831dcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:33:53,874::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:33:53,875::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:33:53,875::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fa557618-77a5-4784-b84f-2e7514831dcf`::Granted request >Thread-34::DEBUG::2014-08-11 14:33:53,876::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fc286eb3-e3b0-4ac9-b6e4-e8564ca2c51f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:33:53,876::task::993::Storage.TaskManager.Task::(_decref) Task=`fc286eb3-e3b0-4ac9-b6e4-e8564ca2c51f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:33:53,897::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862659538944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:53,897::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc286eb3-e3b0-4ac9-b6e4-e8564ca2c51f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862659538944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:33:53,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc286eb3-e3b0-4ac9-b6e4-e8564ca2c51f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:33:53,898::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:33:53,898::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:33:53,898::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:33:53,899::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:33:53,899::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:33:53,899::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:33:53,900::task::993::Storage.TaskManager.Task::(_decref) Task=`fc286eb3-e3b0-4ac9-b6e4-e8564ca2c51f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:33:55,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:55,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013193 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:33:57,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:57,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133287 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:58,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:58,717::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:33:58,718::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:33:58,732::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:58,732::__init__::232::IOProcess::(_processLogs) (5273) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:33:58,733::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:33:58,750::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421385 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:33:58,752::__init__::232::IOProcess::(_processLogs) (5274) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:33:58,753::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:58,754::__init__::232::IOProcess::(_processLogs) (5275) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:58,754::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:58,755::__init__::232::IOProcess::(_processLogs) (5276) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:58,755::__init__::232::IOProcess::(_processLogs) (5276) Queuing response >Thread-14::DEBUG::2014-08-11 14:33:58,756::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:33:58,756::__init__::232::IOProcess::(_processLogs) (5277) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:33:58,757::__init__::232::IOProcess::(_processLogs) (5277) Queuing response >Dummy-33::DEBUG::2014-08-11 14:33:59,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:33:59,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192334 s, 53.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:01,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:01,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133782 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:03,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:03,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131927 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:03,725::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:03,727::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:03,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:34:03,912::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:34:03,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`dce463cd-e78c-4948-aa61-8e45c59c5889`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:03,913::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:34:03,925::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:03,926::task::1191::Storage.TaskManager.Task::(prepare) Task=`dce463cd-e78c-4948-aa61-8e45c59c5889`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:03,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`dce463cd-e78c-4948-aa61-8e45c59c5889`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:03,926::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:03,926::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:03,927::task::993::Storage.TaskManager.Task::(_decref) Task=`dce463cd-e78c-4948-aa61-8e45c59c5889`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:34:03,942::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5c7e3b8-d4cb-4158-992a-f459abdf7c77`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:03,942::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-34::DEBUG::2014-08-11 14:34:03,944::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::INFO::2014-08-11 14:34:03,945::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421385', 'lastCheck': '5.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:34:03,946::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5c7e3b8-d4cb-4158-992a-f459abdf7c77`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421385', 'lastCheck': '5.2', 'valid': True}} >Thread-34::DEBUG::2014-08-11 14:34:03,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd0399fa-e5d3-45b9-a8ba-a01af02de962`::moving from state init -> state preparing >Thread-95::DEBUG::2014-08-11 14:34:03,948::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5c7e3b8-d4cb-4158-992a-f459abdf7c77`::moving from state preparing -> state finished >Thread-34::INFO::2014-08-11 14:34:03,948::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::DEBUG::2014-08-11 14:34:03,949::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:03,950::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:03,952::task::993::Storage.TaskManager.Task::(_decref) Task=`b5c7e3b8-d4cb-4158-992a-f459abdf7c77`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:34:03,952::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1ccd9a9b-31de-497b-a8a4-38a83fbf4a1f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:34:03,953::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:34:03,956::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:34:03,957::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1ccd9a9b-31de-497b-a8a4-38a83fbf4a1f`::Granted request >Thread-34::DEBUG::2014-08-11 14:34:03,960::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cd0399fa-e5d3-45b9-a8ba-a01af02de962`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:34:03,961::task::993::Storage.TaskManager.Task::(_decref) Task=`cd0399fa-e5d3-45b9-a8ba-a01af02de962`::ref 1 aborting False >Thread-95::DEBUG::2014-08-11 14:34:04,010::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:34:04,035::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862384549888', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:04,037::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd0399fa-e5d3-45b9-a8ba-a01af02de962`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862384549888', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:04,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd0399fa-e5d3-45b9-a8ba-a01af02de962`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:34:04,038::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:34:04,038::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:34:04,039::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:34:04,040::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:34:04,040::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:34:04,040::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:34:04,041::task::993::Storage.TaskManager.Task::(_decref) Task=`cd0399fa-e5d3-45b9-a8ba-a01af02de962`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:34:05,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:05,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131776 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:07,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:07,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01335 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:08,732::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:08,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:08,735::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:34:08,759::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:08,759::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:08,759::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:08,760::__init__::232::IOProcess::(_processLogs) (5278) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:08,760::__init__::232::IOProcess::(_processLogs) DEBUG|(5278) Queuing response >Thread-16::DEBUG::2014-08-11 14:34:08,762::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:34:08,781::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418783 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:08,783::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:08,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:08,783::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:08,783::__init__::232::IOProcess::(_processLogs) (5279) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:08,784::__init__::232::IOProcess::(_processLogs) DEBUG|(5279) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:08,785::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:08,785::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:08,786::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:08,786::__init__::232::IOProcess::(_processLogs) DEBUG|(5280) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:08,786::__init__::232::IOProcess::(_processLogs) (5280) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:08,787::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:08,787::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:08,787::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:08,788::__init__::232::IOProcess::(_processLogs) DEBUG|(5281) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:08,788::__init__::232::IOProcess::(_processLogs) (5281) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:08,789::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:08,789::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:08,789::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:08,790::__init__::232::IOProcess::(_processLogs) DEBUG|(5282) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:08,790::__init__::232::IOProcess::(_processLogs) (5282) Queuing response >Dummy-33::DEBUG::2014-08-11 14:34:09,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:09,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132942 s, 77.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:34:11,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`651067ee-bc9e-4117-beee-c43c7460e72c`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:34:11,238::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:34:11,239::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:11,239::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:11,239::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:11,239::__init__::232::IOProcess::(_processLogs) (5283) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:34:11,240::__init__::232::IOProcess::(_processLogs) DEBUG|(5283) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:11,241::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:11,243::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:11,243::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:11,244::__init__::232::IOProcess::(_processLogs) (5284) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:34:11,244::__init__::232::IOProcess::(_processLogs) (5284) Queuing response >Thread-55::INFO::2014-08-11 14:34:11,243::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:34:11,244::task::1191::Storage.TaskManager.Task::(prepare) Task=`651067ee-bc9e-4117-beee-c43c7460e72c`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:34:11,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`651067ee-bc9e-4117-beee-c43c7460e72c`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:34:11,245::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:34:11,245::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:34:11,245::task::993::Storage.TaskManager.Task::(_decref) Task=`651067ee-bc9e-4117-beee-c43c7460e72c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:34:11,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:11,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130936 s, 78.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:13,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:13,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132642 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:13,742::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:13,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:13,744::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:34:14,054::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-95::DEBUG::2014-08-11 14:34:14,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`3edd45fb-2296-4701-ac68-8cc5eb370d64`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:14,055::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:34:14,068::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:14,068::task::1191::Storage.TaskManager.Task::(prepare) Task=`3edd45fb-2296-4701-ac68-8cc5eb370d64`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:14,068::task::595::Storage.TaskManager.Task::(_updateState) Task=`3edd45fb-2296-4701-ac68-8cc5eb370d64`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:14,069::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:14,069::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:14,069::task::993::Storage.TaskManager.Task::(_decref) Task=`3edd45fb-2296-4701-ac68-8cc5eb370d64`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:34:14,080::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-34::DEBUG::2014-08-11 14:34:14,081::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bce2c9b-9be9-40e0-a7e8-b9c701f689db`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:34:14,081::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:34:14,082::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`23527324-25b4-4061-aa91-7e71efe4ba0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:34:14,082::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:34:14,083::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:34:14,083::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`23527324-25b4-4061-aa91-7e71efe4ba0c`::Granted request >Thread-34::DEBUG::2014-08-11 14:34:14,083::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0bce2c9b-9be9-40e0-a7e8-b9c701f689db`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:34:14,084::task::993::Storage.TaskManager.Task::(_decref) Task=`0bce2c9b-9be9-40e0-a7e8-b9c701f689db`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:34:14,112::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862383960064', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:14,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`0bce2c9b-9be9-40e0-a7e8-b9c701f689db`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862383960064', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:14,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bce2c9b-9be9-40e0-a7e8-b9c701f689db`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:34:14,113::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:34:14,113::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:34:14,113::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:34:14,114::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:34:14,114::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:34:14,114::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:34:14,114::task::993::Storage.TaskManager.Task::(_decref) Task=`0bce2c9b-9be9-40e0-a7e8-b9c701f689db`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:34:15,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:15,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132147 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:17,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:17,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132343 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:18,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:18,751::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:18,752::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:34:18,792::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:18,792::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:18,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:18,793::__init__::232::IOProcess::(_processLogs) (5285) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:34:18,793::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:34:18,811::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000431537 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:18,813::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:18,813::__init__::232::IOProcess::(_processLogs) (5286) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:18,814::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:18,814::__init__::232::IOProcess::(_processLogs) (5287) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:18,815::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:18,816::__init__::232::IOProcess::(_processLogs) (5288) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:18,816::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:18,817::__init__::232::IOProcess::(_processLogs) (5289) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:18,817::__init__::232::IOProcess::(_processLogs) (5289) Queuing response >Thread-95::DEBUG::2014-08-11 14:34:19,127::task::595::Storage.TaskManager.Task::(_updateState) Task=`6aafe44b-5d0e-453d-91dd-e3f9e6f7876c`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:19,128::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:34:19,128::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000431537', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:34:19,128::task::1191::Storage.TaskManager.Task::(prepare) Task=`6aafe44b-5d0e-453d-91dd-e3f9e6f7876c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000431537', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:34:19,129::task::595::Storage.TaskManager.Task::(_updateState) Task=`6aafe44b-5d0e-453d-91dd-e3f9e6f7876c`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:19,129::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:19,129::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:19,129::task::993::Storage.TaskManager.Task::(_decref) Task=`6aafe44b-5d0e-453d-91dd-e3f9e6f7876c`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:34:19,148::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:34:19,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:19,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013231 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:21,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:21,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132667 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:23,756::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:23,757::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:23,758::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:34:23,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:23,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132178 s, 77.5 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:34:24,127::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:34:24,127::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfeae90f-f035-4f2c-b2c4-cbe7709fd2c8`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:24,128::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:34:24,140::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:24,141::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfeae90f-f035-4f2c-b2c4-cbe7709fd2c8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:24,141::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfeae90f-f035-4f2c-b2c4-cbe7709fd2c8`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:24,141::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:24,141::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:24,142::task::993::Storage.TaskManager.Task::(_decref) Task=`bfeae90f-f035-4f2c-b2c4-cbe7709fd2c8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:34:24,153::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:34:24,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`037e8721-9405-482a-991d-588fd1b276c4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:34:24,154::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:34:24,154::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d24b74dc-6467-466e-87ac-2a96b70a1678`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:34:24,155::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:34:24,155::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:34:24,155::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d24b74dc-6467-466e-87ac-2a96b70a1678`::Granted request >Thread-34::DEBUG::2014-08-11 14:34:24,156::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`037e8721-9405-482a-991d-588fd1b276c4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:34:24,156::task::993::Storage.TaskManager.Task::(_decref) Task=`037e8721-9405-482a-991d-588fd1b276c4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:34:24,176::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862383501312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:24,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`037e8721-9405-482a-991d-588fd1b276c4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862383501312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:24,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`037e8721-9405-482a-991d-588fd1b276c4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:34:24,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:34:24,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:34:24,177::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:34:24,178::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:34:24,178::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:34:24,178::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:34:24,178::task::993::Storage.TaskManager.Task::(_decref) Task=`037e8721-9405-482a-991d-588fd1b276c4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:34:25,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:25,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132575 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:27,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:28,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133363 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:28,766::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:28,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:34:28,819::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:28,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:28,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:34:28,851::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:34:28,851::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:28,871::__init__::232::IOProcess::(_processLogs) (5290) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:28,871::__init__::232::IOProcess::(_processLogs) (5290) Queuing response >Thread-16::DEBUG::2014-08-11 14:34:28,872::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000504881 s, 901 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:28,873::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:28,874::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:28,874::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:28,874::__init__::232::IOProcess::(_processLogs) (5291) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:28,875::__init__::232::IOProcess::(_processLogs) DEBUG|(5291) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:28,876::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:28,876::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:28,876::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:28,877::__init__::232::IOProcess::(_processLogs) (5292) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:28,877::__init__::232::IOProcess::(_processLogs) DEBUG|(5292) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:28,878::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:28,878::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:28,878::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:28,879::__init__::232::IOProcess::(_processLogs) (5293) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:28,879::__init__::232::IOProcess::(_processLogs) DEBUG|(5293) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:28,880::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:28,880::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:28,881::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:28,881::__init__::232::IOProcess::(_processLogs) (5294) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:28,882::__init__::232::IOProcess::(_processLogs) (5294) Queuing response >Dummy-33::DEBUG::2014-08-11 14:34:30,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:30,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013328 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:32,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:32,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273772 s, 37.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:33,859::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:33,860::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:33,861::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:34:34,192::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:34:34,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`364b6491-07d2-499d-b46e-20103e1b19ad`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:34,193::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:34:34,205::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:34,206::task::1191::Storage.TaskManager.Task::(prepare) Task=`364b6491-07d2-499d-b46e-20103e1b19ad`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:34,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`364b6491-07d2-499d-b46e-20103e1b19ad`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:34,206::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:34,207::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:34,207::task::993::Storage.TaskManager.Task::(_decref) Task=`364b6491-07d2-499d-b46e-20103e1b19ad`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:34:34,219::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:34:34,219::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd21e374-dec0-47fa-bab2-954921fcb594`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:34:34,220::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:34:34,220::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d2d8c600-f411-4ae5-a9e8-9cbef20ac5a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:34:34,221::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:34:34,221::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:34:34,221::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d2d8c600-f411-4ae5-a9e8-9cbef20ac5a8`::Granted request >Thread-34::DEBUG::2014-08-11 14:34:34,222::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dd21e374-dec0-47fa-bab2-954921fcb594`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:34:34,222::task::993::Storage.TaskManager.Task::(_decref) Task=`dd21e374-dec0-47fa-bab2-954921fcb594`::ref 1 aborting False >Thread-95::DEBUG::2014-08-11 14:34:34,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`1abc2983-b2f7-4dd0-b905-87ebe89e57a2`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:34,240::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:34:34,241::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000504881', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:34:34,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`1abc2983-b2f7-4dd0-b905-87ebe89e57a2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000504881', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:34:34,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`1abc2983-b2f7-4dd0-b905-87ebe89e57a2`::moving from state preparing -> state finished >Thread-34::INFO::2014-08-11 14:34:34,243::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862328909824', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-95::DEBUG::2014-08-11 14:34:34,243::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-34::DEBUG::2014-08-11 14:34:34,244::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd21e374-dec0-47fa-bab2-954921fcb594`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1862328909824', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-95::DEBUG::2014-08-11 14:34:34,244::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:34:34,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd21e374-dec0-47fa-bab2-954921fcb594`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:34,246::task::993::Storage.TaskManager.Task::(_decref) Task=`1abc2983-b2f7-4dd0-b905-87ebe89e57a2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:34:34,246::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:34:34,247::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:34:34,249::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:34:34,252::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:34:34,252::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:34:34,253::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:34:34,255::task::993::Storage.TaskManager.Task::(_decref) Task=`dd21e374-dec0-47fa-bab2-954921fcb594`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:34:34,278::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:34:34,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:34,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0282582 s, 36.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:36,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:36,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013367 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:38,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:38,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131693 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:38,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:38,870::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:34:38,883::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:38,949::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:34:38,950::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:38,952::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:34:38,969::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:38,971::__init__::232::IOProcess::(_processLogs) (5295) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:38,972::__init__::232::IOProcess::(_processLogs) (5295) Queuing response >Thread-16::DEBUG::2014-08-11 14:34:38,972::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00129767 s, 351 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:38,973::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:34:38,974::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:38,974::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:38,974::__init__::232::IOProcess::(_processLogs) (5296) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:38,975::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:38,976::__init__::232::IOProcess::(_processLogs) (5297) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:38,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:38,977::__init__::232::IOProcess::(_processLogs) (5298) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:38,977::__init__::232::IOProcess::(_processLogs) (5298) Queuing response >Thread-14::DEBUG::2014-08-11 14:34:38,978::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:38,978::__init__::232::IOProcess::(_processLogs) (5299) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:38,978::__init__::232::IOProcess::(_processLogs) (5299) Queuing response >Dummy-33::DEBUG::2014-08-11 14:34:40,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:40,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132164 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:42,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:42,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131195 s, 78.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:43,977::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:43,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:44,047::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:34:44,278::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:34:44,279::task::595::Storage.TaskManager.Task::(_updateState) Task=`be82bfac-c1ff-48a5-977e-69e3a1b63b37`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:44,279::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:34:44,292::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:44,293::task::1191::Storage.TaskManager.Task::(prepare) Task=`be82bfac-c1ff-48a5-977e-69e3a1b63b37`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:44,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`be82bfac-c1ff-48a5-977e-69e3a1b63b37`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:44,293::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:44,293::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:44,294::task::993::Storage.TaskManager.Task::(_decref) Task=`be82bfac-c1ff-48a5-977e-69e3a1b63b37`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:34:44,305::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:34:44,305::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6e08888-164e-4c5f-a459-17c6d01acb0f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:34:44,305::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:34:44,306::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`79b1365b-890c-4b92-aafb-ce70b1a7cd04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:34:44,306::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:34:44,307::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:34:44,307::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`79b1365b-890c-4b92-aafb-ce70b1a7cd04`::Granted request >Thread-34::DEBUG::2014-08-11 14:34:44,308::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d6e08888-164e-4c5f-a459-17c6d01acb0f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:34:44,308::task::993::Storage.TaskManager.Task::(_decref) Task=`d6e08888-164e-4c5f-a459-17c6d01acb0f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:34:44,327::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861944344576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:44,328::task::1191::Storage.TaskManager.Task::(prepare) Task=`d6e08888-164e-4c5f-a459-17c6d01acb0f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861944344576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:44,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6e08888-164e-4c5f-a459-17c6d01acb0f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:34:44,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:34:44,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:34:44,329::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:34:44,329::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:34:44,329::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:34:44,330::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:34:44,330::task::993::Storage.TaskManager.Task::(_decref) Task=`d6e08888-164e-4c5f-a459-17c6d01acb0f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:34:44,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:44,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131598 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:46,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:46,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013358 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:48,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:48,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132426 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:48,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:48,982::__init__::232::IOProcess::(_processLogs) (5300) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:34:48,983::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:34:49,047::__init__::232::IOProcess::(_processLogs) (5300) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:49,066::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:34:49,068::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000404503 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:49,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:49,072::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:49,073::__init__::232::IOProcess::(_processLogs) (5301) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:49,075::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:34:49,076::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:49,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:49,079::__init__::232::IOProcess::(_processLogs) (5302) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:49,081::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:34:49,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:49,083::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:34:49,085::__init__::232::IOProcess::(_processLogs) (5303) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:49,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:49,087::__init__::232::IOProcess::(_processLogs) (5304) Got request for method 'access' >Thread-95::DEBUG::2014-08-11 14:34:49,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9a6ac9e-e2ee-4af3-976f-02b53b6da8a6`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:49,365::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:34:49,365::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000404503', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:34:49,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9a6ac9e-e2ee-4af3-976f-02b53b6da8a6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000404503', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:34:49,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9a6ac9e-e2ee-4af3-976f-02b53b6da8a6`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:49,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:49,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:49,367::task::993::Storage.TaskManager.Task::(_decref) Task=`d9a6ac9e-e2ee-4af3-976f-02b53b6da8a6`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:34:49,386::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:34:50,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:51,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132431 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:53,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:53,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131478 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:54,091::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:54,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:54,147::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:34:54,342::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:34:54,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`765e4b04-d6e3-4c5e-8e42-565791762a47`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:34:54,343::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:34:54,357::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:54,357::task::1191::Storage.TaskManager.Task::(prepare) Task=`765e4b04-d6e3-4c5e-8e42-565791762a47`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:34:54,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`765e4b04-d6e3-4c5e-8e42-565791762a47`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:34:54,357::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:34:54,358::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:34:54,358::task::993::Storage.TaskManager.Task::(_decref) Task=`765e4b04-d6e3-4c5e-8e42-565791762a47`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:34:54,368::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:34:54,368::task::595::Storage.TaskManager.Task::(_updateState) Task=`a258d0a6-7eb9-4d92-ba68-c9fdcd804ec2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:34:54,369::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:34:54,369::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`90901142-754f-40f7-a031-b9517b73a4ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:34:54,370::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:34:54,370::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:34:54,370::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`90901142-754f-40f7-a031-b9517b73a4ff`::Granted request >Thread-34::DEBUG::2014-08-11 14:34:54,371::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a258d0a6-7eb9-4d92-ba68-c9fdcd804ec2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:34:54,371::task::993::Storage.TaskManager.Task::(_decref) Task=`a258d0a6-7eb9-4d92-ba68-c9fdcd804ec2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:34:54,391::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861944147968', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:54,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`a258d0a6-7eb9-4d92-ba68-c9fdcd804ec2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861944147968', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:34:54,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`a258d0a6-7eb9-4d92-ba68-c9fdcd804ec2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:34:54,392::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:34:54,392::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:34:54,392::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:34:54,393::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:34:54,393::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:34:54,393::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:34:54,394::task::993::Storage.TaskManager.Task::(_decref) Task=`a258d0a6-7eb9-4d92-ba68-c9fdcd804ec2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:34:55,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:55,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140808 s, 72.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:34:57,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:57,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131487 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:59,089::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:34:59,089::__init__::232::IOProcess::(_processLogs) (5305) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:34:59,090::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:34:59,109::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042755 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:34:59,110::__init__::232::IOProcess::(_processLogs) (5306) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:34:59,148::__init__::232::IOProcess::(_processLogs) (5307) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:59,149::__init__::232::IOProcess::(_processLogs) (5308) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:34:59,150::__init__::232::IOProcess::(_processLogs) (5309) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:59,153::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:59,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:34:59,155::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:34:59,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:34:59,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132087 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:01,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:01,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133452 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:03,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:03,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131265 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:04,162::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:04,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:04,247::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:04,406::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:35:04,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`0790686b-9a49-4d41-858c-ebf716881d9b`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:04,407::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:35:04,420::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:04,420::task::1191::Storage.TaskManager.Task::(prepare) Task=`0790686b-9a49-4d41-858c-ebf716881d9b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:04,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`0790686b-9a49-4d41-858c-ebf716881d9b`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:04,420::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:04,421::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:04,421::task::993::Storage.TaskManager.Task::(_decref) Task=`0790686b-9a49-4d41-858c-ebf716881d9b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:35:04,434::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:35:04,434::task::595::Storage.TaskManager.Task::(_updateState) Task=`163c351d-f9e2-4a3d-bc67-89ea878cd586`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:35:04,435::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:35:04,435::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9916aa0-7e11-48d1-be85-a7d3176c351d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:35:04,436::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:35:04,436::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:35:04,436::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9916aa0-7e11-48d1-be85-a7d3176c351d`::Granted request >Thread-34::DEBUG::2014-08-11 14:35:04,437::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`163c351d-f9e2-4a3d-bc67-89ea878cd586`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:35:04,437::task::993::Storage.TaskManager.Task::(_decref) Task=`163c351d-f9e2-4a3d-bc67-89ea878cd586`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:35:04,457::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861782732800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:04,457::task::1191::Storage.TaskManager.Task::(prepare) Task=`163c351d-f9e2-4a3d-bc67-89ea878cd586`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861782732800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:04,458::task::595::Storage.TaskManager.Task::(_updateState) Task=`163c351d-f9e2-4a3d-bc67-89ea878cd586`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:35:04,458::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:35:04,458::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:35:04,458::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:35:04,459::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:35:04,459::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:35:04,459::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:35:04,459::task::993::Storage.TaskManager.Task::(_decref) Task=`163c351d-f9e2-4a3d-bc67-89ea878cd586`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:35:04,478::task::595::Storage.TaskManager.Task::(_updateState) Task=`3990b470-00ca-49f3-b209-493911e15f7e`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:04,478::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:35:04,479::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042755', 'lastCheck': '5.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:04,479::task::1191::Storage.TaskManager.Task::(prepare) Task=`3990b470-00ca-49f3-b209-493911e15f7e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042755', 'lastCheck': '5.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:04,479::task::595::Storage.TaskManager.Task::(_updateState) Task=`3990b470-00ca-49f3-b209-493911e15f7e`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:04,479::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:04,479::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:04,480::task::993::Storage.TaskManager.Task::(_decref) Task=`3990b470-00ca-49f3-b209-493911e15f7e`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:35:04,504::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:35:05,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:05,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131816 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:07,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:07,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013146 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:09,152::__init__::232::IOProcess::(_processLogs) (5310) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:35:09,153::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:35:09,172::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425169 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:09,173::__init__::232::IOProcess::(_processLogs) (5311) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:09,248::__init__::232::IOProcess::(_processLogs) (5312) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:09,249::__init__::232::IOProcess::(_processLogs) (5313) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:09,250::__init__::232::IOProcess::(_processLogs) (5314) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:09,254::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:09,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:09,256::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:35:09,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:09,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130961 s, 78.2 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:35:11,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`0eb8b3d8-789c-4003-9ce4-357b74af2fea`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:35:11,352::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:35:11,353::__init__::232::IOProcess::(_processLogs) (5315) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:35:11,354::__init__::232::IOProcess::(_processLogs) (5316) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:35:11,355::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:35:11,355::task::1191::Storage.TaskManager.Task::(prepare) Task=`0eb8b3d8-789c-4003-9ce4-357b74af2fea`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:35:11,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`0eb8b3d8-789c-4003-9ce4-357b74af2fea`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:35:11,356::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:35:11,356::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:35:11,356::task::993::Storage.TaskManager.Task::(_decref) Task=`0eb8b3d8-789c-4003-9ce4-357b74af2fea`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:35:11,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:11,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132444 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:13,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:13,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133366 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:14,263::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:14,264::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:14,347::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:14,475::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:35:14,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fad60d6-438c-4aea-a301-a8d38e76c847`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:14,476::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:35:14,489::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:14,489::task::1191::Storage.TaskManager.Task::(prepare) Task=`8fad60d6-438c-4aea-a301-a8d38e76c847`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:14,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fad60d6-438c-4aea-a301-a8d38e76c847`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:14,490::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:14,490::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:14,490::task::993::Storage.TaskManager.Task::(_decref) Task=`8fad60d6-438c-4aea-a301-a8d38e76c847`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:35:14,500::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:35:14,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd4f960b-09b7-42e6-848c-2f8a7b05c89a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:35:14,501::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:35:14,502::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`05734b0d-d322-4a10-84bc-11c0491c1ffe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:35:14,502::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:35:14,502::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:35:14,503::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`05734b0d-d322-4a10-84bc-11c0491c1ffe`::Granted request >Thread-34::DEBUG::2014-08-11 14:35:14,503::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dd4f960b-09b7-42e6-848c-2f8a7b05c89a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:35:14,503::task::993::Storage.TaskManager.Task::(_decref) Task=`dd4f960b-09b7-42e6-848c-2f8a7b05c89a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:35:14,523::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861517705216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:14,524::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd4f960b-09b7-42e6-848c-2f8a7b05c89a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861517705216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:14,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd4f960b-09b7-42e6-848c-2f8a7b05c89a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:35:14,524::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:35:14,524::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:35:14,525::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:35:14,525::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:35:14,525::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:35:14,525::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:35:14,526::task::993::Storage.TaskManager.Task::(_decref) Task=`dd4f960b-09b7-42e6-848c-2f8a7b05c89a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:35:15,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:15,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132369 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:17,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:17,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132403 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:19,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:19,252::__init__::232::IOProcess::(_processLogs) (5317) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:35:19,253::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:35:19,271::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422795 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:19,272::__init__::232::IOProcess::(_processLogs) (5318) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:19,348::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:19,348::__init__::232::IOProcess::(_processLogs) (5319) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:19,349::__init__::232::IOProcess::(_processLogs) (5320) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:19,350::__init__::232::IOProcess::(_processLogs) (5321) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:19,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:19,354::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:19,355::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:19,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`151131eb-e46b-4701-b7eb-54d525412356`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:19,609::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:35:19,610::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422795', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:19,611::task::1191::Storage.TaskManager.Task::(prepare) Task=`151131eb-e46b-4701-b7eb-54d525412356`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422795', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:19,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`151131eb-e46b-4701-b7eb-54d525412356`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:19,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:19,613::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:19,615::task::993::Storage.TaskManager.Task::(_decref) Task=`151131eb-e46b-4701-b7eb-54d525412356`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:35:19,638::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:35:19,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:19,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134016 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:21,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:21,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013194 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:23,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:23,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131817 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:24,360::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:24,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:24,448::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:24,537::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:35:24,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`6caf32c7-30d7-45f0-a40c-47b8a224c0ac`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:24,539::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:35:24,551::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:24,552::task::1191::Storage.TaskManager.Task::(prepare) Task=`6caf32c7-30d7-45f0-a40c-47b8a224c0ac`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:24,552::task::595::Storage.TaskManager.Task::(_updateState) Task=`6caf32c7-30d7-45f0-a40c-47b8a224c0ac`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:24,552::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:24,552::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:24,552::task::993::Storage.TaskManager.Task::(_decref) Task=`6caf32c7-30d7-45f0-a40c-47b8a224c0ac`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:35:24,562::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:35:24,563::task::595::Storage.TaskManager.Task::(_updateState) Task=`316508f7-81b2-4f40-a320-8e8243695a56`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:35:24,563::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:35:24,564::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`63d16128-9ddf-41f7-9d04-8f34d9430ece`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:35:24,564::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:35:24,564::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:35:24,565::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`63d16128-9ddf-41f7-9d04-8f34d9430ece`::Granted request >Thread-34::DEBUG::2014-08-11 14:35:24,565::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`316508f7-81b2-4f40-a320-8e8243695a56`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:35:24,565::task::993::Storage.TaskManager.Task::(_decref) Task=`316508f7-81b2-4f40-a320-8e8243695a56`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:35:24,585::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861517770752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:24,586::task::1191::Storage.TaskManager.Task::(prepare) Task=`316508f7-81b2-4f40-a320-8e8243695a56`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861517770752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:24,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`316508f7-81b2-4f40-a320-8e8243695a56`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:35:24,586::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:35:24,586::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:35:24,587::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:35:24,587::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:35:24,587::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:35:24,587::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:35:24,588::task::993::Storage.TaskManager.Task::(_decref) Task=`316508f7-81b2-4f40-a320-8e8243695a56`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:35:25,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:25,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132289 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:27,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:28,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132247 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:29,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:29,353::__init__::232::IOProcess::(_processLogs) (5322) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:35:29,353::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:35:29,371::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427832 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:29,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:29,449::__init__::232::IOProcess::(_processLogs) (5323) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:29,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:29,451::__init__::232::IOProcess::(_processLogs) (5324) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:29,452::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:29,452::__init__::232::IOProcess::(_processLogs) (5325) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:29,452::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5325) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:29,453::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:29,453::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:29,454::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:29,454::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5326) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:29,456::__init__::232::IOProcess::(_processLogs) (5326) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:29,458::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:29,459::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:29,460::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:35:30,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:30,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013301 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:32,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:32,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272861 s, 37.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:34,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:34,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132241 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:34,465::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:34,467::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:34,468::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:34,600::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:35:34,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`1418c95f-a2ef-4d08-95ab-f13c4712788f`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:34,601::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:35:34,614::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:34,614::task::1191::Storage.TaskManager.Task::(prepare) Task=`1418c95f-a2ef-4d08-95ab-f13c4712788f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:34,615::task::595::Storage.TaskManager.Task::(_updateState) Task=`1418c95f-a2ef-4d08-95ab-f13c4712788f`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:34,615::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:34,615::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:34,615::task::993::Storage.TaskManager.Task::(_decref) Task=`1418c95f-a2ef-4d08-95ab-f13c4712788f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:35:34,626::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:35:34,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc3e0195-6355-41b0-a384-b03a0b449ebf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:35:34,627::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:35:34,628::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fb95aad4-b94b-435d-a385-9465ea3b7129`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:35:34,628::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:35:34,629::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:35:34,629::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fb95aad4-b94b-435d-a385-9465ea3b7129`::Granted request >Thread-34::DEBUG::2014-08-11 14:35:34,629::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bc3e0195-6355-41b0-a384-b03a0b449ebf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:35:34,630::task::993::Storage.TaskManager.Task::(_decref) Task=`bc3e0195-6355-41b0-a384-b03a0b449ebf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:35:34,649::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861516460032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:34,650::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc3e0195-6355-41b0-a384-b03a0b449ebf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861516460032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:34,650::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc3e0195-6355-41b0-a384-b03a0b449ebf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:35:34,650::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:35:34,651::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:35:34,651::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:35:34,651::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:35:34,651::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:35:34,652::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:35:34,652::task::993::Storage.TaskManager.Task::(_decref) Task=`bc3e0195-6355-41b0-a384-b03a0b449ebf`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:35:34,726::task::595::Storage.TaskManager.Task::(_updateState) Task=`86c1ac58-4731-4b1c-9e58-713e32440857`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:34,727::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:35:34,727::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427832', 'lastCheck': '5.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:34,727::task::1191::Storage.TaskManager.Task::(prepare) Task=`86c1ac58-4731-4b1c-9e58-713e32440857`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427832', 'lastCheck': '5.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:34,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`86c1ac58-4731-4b1c-9e58-713e32440857`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:34,728::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:34,728::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:34,728::task::993::Storage.TaskManager.Task::(_decref) Task=`86c1ac58-4731-4b1c-9e58-713e32440857`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:35:34,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:35:36,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:36,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132387 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:38,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:38,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132218 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:39,459::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:39,459::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:39,459::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:39,460::__init__::232::IOProcess::(_processLogs) (5327) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:39,460::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5327) Queuing response >Thread-16::DEBUG::2014-08-11 14:35:39,461::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:39,479::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:39,481::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:35:39,483::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422402 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:39,551::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:39,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:39,553::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:35:39,554::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:39,555::__init__::232::IOProcess::(_processLogs) (5328) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:39,558::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5328) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:39,559::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:39,560::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:39,560::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:39,560::__init__::232::IOProcess::(_processLogs) (5329) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:39,560::__init__::232::IOProcess::(_processLogs) (5329) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:39,561::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:39,562::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:39,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:39,563::__init__::232::IOProcess::(_processLogs) (5330) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:39,563::__init__::232::IOProcess::(_processLogs) (5330) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:39,563::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:39,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:39,564::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:39,565::__init__::232::IOProcess::(_processLogs) (5331) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:39,566::__init__::232::IOProcess::(_processLogs) (5331) Queuing response >Dummy-33::DEBUG::2014-08-11 14:35:40,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:40,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133206 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:42,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:42,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131969 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:44,565::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:44,567::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:35:44,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-95::DEBUG::2014-08-11 14:35:44,670::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:44,671::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:44,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`566c4db6-bad2-4b31-8140-642d05ba80d2`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:44,676::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 14:35:44,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165507 s, 61.9 MB/s\n'; <rc> = 0 >Thread-95::INFO::2014-08-11 14:35:44,694::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:44,695::task::1191::Storage.TaskManager.Task::(prepare) Task=`566c4db6-bad2-4b31-8140-642d05ba80d2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:44,696::task::595::Storage.TaskManager.Task::(_updateState) Task=`566c4db6-bad2-4b31-8140-642d05ba80d2`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:44,697::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:44,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:44,698::task::993::Storage.TaskManager.Task::(_decref) Task=`566c4db6-bad2-4b31-8140-642d05ba80d2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:35:44,711::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:35:44,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`92e1279c-e9a7-443b-8950-dab6a6b3b5c5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:35:44,715::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:35:44,720::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30820dc9-ff62-4ebf-9fa0-c59f94cfa64e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:35:44,721::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:35:44,723::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:35:44,724::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30820dc9-ff62-4ebf-9fa0-c59f94cfa64e`::Granted request >Thread-34::DEBUG::2014-08-11 14:35:44,727::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`92e1279c-e9a7-443b-8950-dab6a6b3b5c5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:35:44,728::task::993::Storage.TaskManager.Task::(_decref) Task=`92e1279c-e9a7-443b-8950-dab6a6b3b5c5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:35:44,755::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861358059520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:44,756::task::1191::Storage.TaskManager.Task::(prepare) Task=`92e1279c-e9a7-443b-8950-dab6a6b3b5c5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861358059520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:44,756::task::595::Storage.TaskManager.Task::(_updateState) Task=`92e1279c-e9a7-443b-8950-dab6a6b3b5c5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:35:44,756::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:35:44,757::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:35:44,757::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:35:44,757::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:35:44,757::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:35:44,758::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:35:44,758::task::993::Storage.TaskManager.Task::(_decref) Task=`92e1279c-e9a7-443b-8950-dab6a6b3b5c5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:35:46,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:46,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131872 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:48,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:48,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134539 s, 76.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:49,567::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:49,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:35:49,568::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:35:49,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:49,667::__init__::232::IOProcess::(_processLogs) (5332) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:49,668::__init__::232::IOProcess::(_processLogs) (5332) Queuing response >Thread-16::DEBUG::2014-08-11 14:35:49,668::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000406307 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:49,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:49,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:49,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:49,670::__init__::232::IOProcess::(_processLogs) (5333) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:49,670::__init__::232::IOProcess::(_processLogs) (5333) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:49,672::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:49,672::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:49,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:49,673::__init__::232::IOProcess::(_processLogs) (5334) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:49,673::__init__::232::IOProcess::(_processLogs) (5334) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:49,674::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:49,674::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:49,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:49,675::__init__::232::IOProcess::(_processLogs) (5335) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:49,675::__init__::232::IOProcess::(_processLogs) (5335) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:49,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:49,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:49,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:49,678::__init__::232::IOProcess::(_processLogs) (5336) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:49,678::__init__::232::IOProcess::(_processLogs) (5336) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:49,681::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:49,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:49,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:49,837::task::595::Storage.TaskManager.Task::(_updateState) Task=`799f7165-cd89-4f9e-9a05-7eae83b50197`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:49,837::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:35:49,838::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406307', 'lastCheck': '0.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:49,838::task::1191::Storage.TaskManager.Task::(prepare) Task=`799f7165-cd89-4f9e-9a05-7eae83b50197`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406307', 'lastCheck': '0.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:35:49,838::task::595::Storage.TaskManager.Task::(_updateState) Task=`799f7165-cd89-4f9e-9a05-7eae83b50197`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:49,839::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:49,839::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:49,839::task::993::Storage.TaskManager.Task::(_decref) Task=`799f7165-cd89-4f9e-9a05-7eae83b50197`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:35:49,860::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:35:50,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:50,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132147 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:52,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:52,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131765 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:54,690::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:54,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:54,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:35:54,772::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:35:54,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2f803cb-e63f-4a13-b3a8-89f6ddec278f`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:35:54,774::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:35:54,787::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:54,787::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2f803cb-e63f-4a13-b3a8-89f6ddec278f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:35:54,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2f803cb-e63f-4a13-b3a8-89f6ddec278f`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:35:54,788::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:35:54,788::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:35:54,788::task::993::Storage.TaskManager.Task::(_decref) Task=`e2f803cb-e63f-4a13-b3a8-89f6ddec278f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:35:54,799::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:35:54,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`75c09fa5-116c-477e-8927-65acbac2c5c4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:35:54,800::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:35:54,801::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c50b75c5-5ad2-4ae7-847c-a5abaab64b83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:35:54,801::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:35:54,801::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:35:54,801::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c50b75c5-5ad2-4ae7-847c-a5abaab64b83`::Granted request >Thread-34::DEBUG::2014-08-11 14:35:54,802::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`75c09fa5-116c-477e-8927-65acbac2c5c4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:35:54,802::task::993::Storage.TaskManager.Task::(_decref) Task=`75c09fa5-116c-477e-8927-65acbac2c5c4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:35:54,822::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861313626112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:54,822::task::1191::Storage.TaskManager.Task::(prepare) Task=`75c09fa5-116c-477e-8927-65acbac2c5c4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861313626112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:35:54,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`75c09fa5-116c-477e-8927-65acbac2c5c4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:35:54,823::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:35:54,823::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:35:54,823::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:35:54,824::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:35:54,824::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:35:54,824::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:35:54,824::task::993::Storage.TaskManager.Task::(_decref) Task=`75c09fa5-116c-477e-8927-65acbac2c5c4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:35:54,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:54,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131804 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:57,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:57,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131832 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:35:59,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:35:59,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132854 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:59,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:59,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:35:59,681::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:35:59,681::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:59,700::__init__::232::IOProcess::(_processLogs) (5337) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:59,701::__init__::232::IOProcess::(_processLogs) (5337) Queuing response >Thread-16::DEBUG::2014-08-11 14:35:59,701::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00072587 s, 627 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:35:59,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:59,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:59,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:59,752::__init__::232::IOProcess::(_processLogs) (5338) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:35:59,753::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5338) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:59,754::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:59,754::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:35:59,755::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:59,755::__init__::232::IOProcess::(_processLogs) (5339) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:59,755::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5339) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:59,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:59,759::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:59,759::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:35:59,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:59,762::__init__::232::IOProcess::(_processLogs) (5340) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:59,765::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:35:59,765::__init__::232::IOProcess::(_processLogs) (5340) Queuing response >Thread-14::DEBUG::2014-08-11 14:35:59,767::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:35:59,769::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:35:59,770::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:35:59,770::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:35:59,771::__init__::232::IOProcess::(_processLogs) (5341) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:35:59,773::__init__::232::IOProcess::(_processLogs) DEBUG|(5341) Queuing response >Dummy-33::DEBUG::2014-08-11 14:36:01,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:01,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013274 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:03,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:03,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132941 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:04,781::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:04,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:04,843::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:36:04,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`90f10fe0-bc31-47ab-baae-10b1426b5d3f`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:04,853::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:04,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:36:04,867::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:04,867::task::1191::Storage.TaskManager.Task::(prepare) Task=`90f10fe0-bc31-47ab-baae-10b1426b5d3f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:04,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`90f10fe0-bc31-47ab-baae-10b1426b5d3f`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:04,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:04,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:04,868::task::993::Storage.TaskManager.Task::(_decref) Task=`90f10fe0-bc31-47ab-baae-10b1426b5d3f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:36:04,879::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:36:04,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3be9633-8eef-42b2-b9a8-18068e9b9ab1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:36:04,880::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:36:04,880::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`65e4547b-b37e-4d2f-a2dd-c201d0f07af7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:36:04,881::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:36:04,881::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:36:04,881::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`65e4547b-b37e-4d2f-a2dd-c201d0f07af7`::Granted request >Thread-34::DEBUG::2014-08-11 14:36:04,882::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f3be9633-8eef-42b2-b9a8-18068e9b9ab1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:36:04,882::task::993::Storage.TaskManager.Task::(_decref) Task=`f3be9633-8eef-42b2-b9a8-18068e9b9ab1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:36:04,902::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861222596608', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:04,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3be9633-8eef-42b2-b9a8-18068e9b9ab1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861222596608', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:04,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3be9633-8eef-42b2-b9a8-18068e9b9ab1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:36:04,903::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:36:04,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:36:04,903::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:36:04,904::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:36:04,904::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:36:04,904::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:36:04,904::task::993::Storage.TaskManager.Task::(_decref) Task=`f3be9633-8eef-42b2-b9a8-18068e9b9ab1`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:36:04,950::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cc61b16-f63c-4a6a-804f-d9df5a74a848`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:04,951::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:36:04,951::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00072587', 'lastCheck': '5.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:04,951::task::1191::Storage.TaskManager.Task::(prepare) Task=`5cc61b16-f63c-4a6a-804f-d9df5a74a848`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00072587', 'lastCheck': '5.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:04,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cc61b16-f63c-4a6a-804f-d9df5a74a848`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:04,952::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:04,952::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:04,952::task::993::Storage.TaskManager.Task::(_decref) Task=`5cc61b16-f63c-4a6a-804f-d9df5a74a848`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:36:04,971::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:36:05,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:05,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131776 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:07,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:07,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132261 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:09,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:09,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132359 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:09,775::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:09,776::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:36:09,777::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:36:09,778::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:09,797::__init__::232::IOProcess::(_processLogs) (5342) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:09,799::__init__::232::IOProcess::(_processLogs) (5342) Queuing response >Thread-16::DEBUG::2014-08-11 14:36:09,800::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00109009 s, 417 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:09,852::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:09,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:09,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:09,854::__init__::232::IOProcess::(_processLogs) (5343) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:09,855::__init__::232::IOProcess::(_processLogs) DEBUG|(5343) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:09,855::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:09,856::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:09,856::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:09,856::__init__::232::IOProcess::(_processLogs) (5344) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:09,857::__init__::232::IOProcess::(_processLogs) DEBUG|(5344) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:09,858::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:09,858::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:09,859::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:09,859::__init__::232::IOProcess::(_processLogs) (5345) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:09,859::__init__::232::IOProcess::(_processLogs) (5345) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:09,860::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:09,862::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:09,863::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:36:09,864::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:09,866::__init__::232::IOProcess::(_processLogs) (5346) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:09,868::__init__::232::IOProcess::(_processLogs) (5346) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:09,869::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:09,870::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:36:11,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:11,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131763 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:36:11,459::task::595::Storage.TaskManager.Task::(_updateState) Task=`57bf7256-f9ed-4886-9e62-a355323c5598`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:36:11,460::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:36:11,461::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:11,461::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:11,462::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:11,462::__init__::232::IOProcess::(_processLogs) (5347) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:36:11,463::__init__::232::IOProcess::(_processLogs) DEBUG|(5347) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:11,464::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:11,464::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:36:11,465::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:36:11,466::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:36:11,466::task::1191::Storage.TaskManager.Task::(prepare) Task=`57bf7256-f9ed-4886-9e62-a355323c5598`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:36:11,467::__init__::232::IOProcess::(_processLogs) (5348) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:36:11,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`57bf7256-f9ed-4886-9e62-a355323c5598`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:36:11,468::__init__::232::IOProcess::(_processLogs) (5348) Queuing response >Thread-55::DEBUG::2014-08-11 14:36:11,469::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:36:11,470::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:36:11,470::task::993::Storage.TaskManager.Task::(_decref) Task=`57bf7256-f9ed-4886-9e62-a355323c5598`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:36:13,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:13,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132696 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:14,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:14,878::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:14,917::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:36:14,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ce66f6f-f578-4823-92e9-ab373798db7d`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:14,953::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:14,954::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:36:14,967::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:14,967::task::1191::Storage.TaskManager.Task::(prepare) Task=`8ce66f6f-f578-4823-92e9-ab373798db7d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:14,968::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ce66f6f-f578-4823-92e9-ab373798db7d`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:14,968::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:14,968::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:14,968::task::993::Storage.TaskManager.Task::(_decref) Task=`8ce66f6f-f578-4823-92e9-ab373798db7d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:36:14,980::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:36:14,981::task::595::Storage.TaskManager.Task::(_updateState) Task=`04b6d7a3-d5f1-4f29-a881-d0668f4a7f98`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:36:14,981::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:36:14,982::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e352d5a8-f2e6-4f78-b8fc-cbcde3f1ee0a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:36:14,982::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:36:14,982::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:36:14,983::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e352d5a8-f2e6-4f78-b8fc-cbcde3f1ee0a`::Granted request >Thread-34::DEBUG::2014-08-11 14:36:14,983::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`04b6d7a3-d5f1-4f29-a881-d0668f4a7f98`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:36:14,983::task::993::Storage.TaskManager.Task::(_decref) Task=`04b6d7a3-d5f1-4f29-a881-d0668f4a7f98`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:36:15,003::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861161058304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:15,003::task::1191::Storage.TaskManager.Task::(prepare) Task=`04b6d7a3-d5f1-4f29-a881-d0668f4a7f98`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861161058304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:15,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`04b6d7a3-d5f1-4f29-a881-d0668f4a7f98`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:36:15,004::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:36:15,004::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:36:15,005::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:36:15,005::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:36:15,005::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:36:15,005::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:36:15,006::task::993::Storage.TaskManager.Task::(_decref) Task=`04b6d7a3-d5f1-4f29-a881-d0668f4a7f98`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:36:15,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:15,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132592 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:17,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:17,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131937 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:19,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:19,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013389 s, 76.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:19,865::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:19,866::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:36:19,867::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:36:19,867::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:19,884::__init__::232::IOProcess::(_processLogs) (5349) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:19,885::__init__::232::IOProcess::(_processLogs) (5349) Queuing response >Thread-16::DEBUG::2014-08-11 14:36:19,886::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000824362 s, 552 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:19,951::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:19,951::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:19,951::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:19,952::__init__::232::IOProcess::(_processLogs) (5350) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:19,952::__init__::232::IOProcess::(_processLogs) DEBUG|(5350) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:19,953::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:19,953::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:19,954::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:19,954::__init__::232::IOProcess::(_processLogs) (5351) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:19,954::__init__::232::IOProcess::(_processLogs) DEBUG|(5351) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:19,955::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:19,955::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:19,955::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:19,956::__init__::232::IOProcess::(_processLogs) (5352) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:19,956::__init__::232::IOProcess::(_processLogs) DEBUG|(5352) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:19,957::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:19,957::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:19,958::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:19,958::__init__::232::IOProcess::(_processLogs) (5353) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:19,959::__init__::232::IOProcess::(_processLogs) (5353) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:19,962::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:19,963::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:19,964::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:20,060::task::595::Storage.TaskManager.Task::(_updateState) Task=`851dfb4d-3f98-41c2-907a-005dcb9f6fee`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:20,060::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:36:20,061::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000824362', 'lastCheck': '0.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:20,061::task::1191::Storage.TaskManager.Task::(prepare) Task=`851dfb4d-3f98-41c2-907a-005dcb9f6fee`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000824362', 'lastCheck': '0.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:20,061::task::595::Storage.TaskManager.Task::(_updateState) Task=`851dfb4d-3f98-41c2-907a-005dcb9f6fee`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:20,061::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:20,061::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:20,062::task::993::Storage.TaskManager.Task::(_decref) Task=`851dfb4d-3f98-41c2-907a-005dcb9f6fee`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:36:20,080::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:36:21,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:21,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133634 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:23,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:23,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132546 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:24,966::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:24,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:25,018::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:36:25,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c52f424-7c66-4827-bf01-0a83c367f99e`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:25,054::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:25,054::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:36:25,068::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:25,068::task::1191::Storage.TaskManager.Task::(prepare) Task=`7c52f424-7c66-4827-bf01-0a83c367f99e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:25,068::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c52f424-7c66-4827-bf01-0a83c367f99e`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:25,068::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:25,069::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:25,069::task::993::Storage.TaskManager.Task::(_decref) Task=`7c52f424-7c66-4827-bf01-0a83c367f99e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:36:25,079::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:36:25,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bbd93bf-3faf-458a-9f01-02b6253da7c0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:36:25,080::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:36:25,081::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d6692a4d-f481-43bc-be0a-e061ee09aabe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:36:25,081::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:36:25,082::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:36:25,082::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d6692a4d-f481-43bc-be0a-e061ee09aabe`::Granted request >Thread-34::DEBUG::2014-08-11 14:36:25,082::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4bbd93bf-3faf-458a-9f01-02b6253da7c0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:36:25,083::task::993::Storage.TaskManager.Task::(_decref) Task=`4bbd93bf-3faf-458a-9f01-02b6253da7c0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:36:25,103::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:25,103::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bbd93bf-3faf-458a-9f01-02b6253da7c0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:25,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bbd93bf-3faf-458a-9f01-02b6253da7c0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:36:25,103::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:36:25,104::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:36:25,104::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:36:25,104::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:36:25,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:36:25,105::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:36:25,105::task::993::Storage.TaskManager.Task::(_decref) Task=`4bbd93bf-3faf-458a-9f01-02b6253da7c0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:36:25,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:25,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132387 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:27,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:27,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132031 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:29,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:29,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132027 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:29,961::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:29,962::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:29,963::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:29,964::__init__::232::IOProcess::(_processLogs) (5354) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:29,966::__init__::232::IOProcess::(_processLogs) DEBUG|(5354) Queuing response >Thread-16::DEBUG::2014-08-11 14:36:29,968::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:36:29,988::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00148158 s, 307 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:30,050::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:30,051::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:30,053::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:30,053::__init__::232::IOProcess::(_processLogs) (5355) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:30,053::__init__::232::IOProcess::(_processLogs) (5355) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:30,054::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:30,055::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:30,055::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:30,055::__init__::232::IOProcess::(_processLogs) (5356) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:30,056::__init__::232::IOProcess::(_processLogs) (5356) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:30,056::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:30,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:30,057::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:30,058::__init__::232::IOProcess::(_processLogs) (5357) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:30,058::__init__::232::IOProcess::(_processLogs) (5357) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:30,058::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:30,059::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:30,060::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:30,060::__init__::232::IOProcess::(_processLogs) (5358) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:30,062::__init__::232::IOProcess::(_processLogs) (5358) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:30,063::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:30,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:30,066::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:36:31,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:32,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0290357 s, 35.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:34,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:34,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131493 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:35,073::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:35,074::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:35,117::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-95::DEBUG::2014-08-11 14:36:35,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`8efcfc95-d87e-43cd-89de-6fc3f82d7c41`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:35,154::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:35,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:36:35,174::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:35,175::task::1191::Storage.TaskManager.Task::(prepare) Task=`8efcfc95-d87e-43cd-89de-6fc3f82d7c41`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:35,175::task::595::Storage.TaskManager.Task::(_updateState) Task=`8efcfc95-d87e-43cd-89de-6fc3f82d7c41`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:35,175::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:35,175::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:35,176::task::993::Storage.TaskManager.Task::(_decref) Task=`8efcfc95-d87e-43cd-89de-6fc3f82d7c41`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:36:35,190::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-34::DEBUG::2014-08-11 14:36:35,190::task::595::Storage.TaskManager.Task::(_updateState) Task=`47d537f3-f84b-461b-9029-b7a01e566f63`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:36:35,191::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:36:35,191::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fa2d95a9-9e45-42ea-8056-7dc85c3b1c09`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:36:35,192::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:36:35,192::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:36:35,192::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fa2d95a9-9e45-42ea-8056-7dc85c3b1c09`::Granted request >Thread-34::DEBUG::2014-08-11 14:36:35,193::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`47d537f3-f84b-461b-9029-b7a01e566f63`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:36:35,193::task::993::Storage.TaskManager.Task::(_decref) Task=`47d537f3-f84b-461b-9029-b7a01e566f63`::ref 1 aborting False >Thread-95::DEBUG::2014-08-11 14:36:35,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d396ce8-762a-4328-b337-60b3a294215e`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:35,202::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:36:35,204::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00148158', 'lastCheck': '5.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:35,206::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d396ce8-762a-4328-b337-60b3a294215e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00148158', 'lastCheck': '5.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:35,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d396ce8-762a-4328-b337-60b3a294215e`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:35,208::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:35,209::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:35,210::task::993::Storage.TaskManager.Task::(_decref) Task=`3d396ce8-762a-4328-b337-60b3a294215e`::ref 0 aborting False >Thread-34::INFO::2014-08-11 14:36:35,244::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:35,244::task::1191::Storage.TaskManager.Task::(prepare) Task=`47d537f3-f84b-461b-9029-b7a01e566f63`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:35,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`47d537f3-f84b-461b-9029-b7a01e566f63`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:36:35,245::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:36:35,245::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:36:35,245::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:36:35,246::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:36:35,246::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:36:35,246::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:36:35,246::task::993::Storage.TaskManager.Task::(_decref) Task=`47d537f3-f84b-461b-9029-b7a01e566f63`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:36:35,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:36:36,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:36,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013127 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:38,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:38,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131706 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:40,061::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:40,062::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:40,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:40,062::__init__::232::IOProcess::(_processLogs) (5359) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:40,063::__init__::232::IOProcess::(_processLogs) DEBUG|(5359) Queuing response >Thread-16::DEBUG::2014-08-11 14:36:40,064::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:36:40,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417099 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:40,083::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:40,151::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:40,152::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:40,153::__init__::232::IOProcess::(_processLogs) (5360) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:40,153::__init__::232::IOProcess::(_processLogs) (5360) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:40,153::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:40,154::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:40,154::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:40,154::__init__::232::IOProcess::(_processLogs) (5361) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:40,155::__init__::232::IOProcess::(_processLogs) DEBUG|(5361) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:40,155::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:40,156::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:40,156::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:40,156::__init__::232::IOProcess::(_processLogs) (5362) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:40,157::__init__::232::IOProcess::(_processLogs) DEBUG|(5362) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:40,158::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:40,158::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:40,158::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:40,158::__init__::232::IOProcess::(_processLogs) (5363) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:40,159::__init__::232::IOProcess::(_processLogs) DEBUG|(5363) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:40,162::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:40,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:40,165::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:36:40,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:40,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131059 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:42,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:42,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133016 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:44,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:44,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159787 s, 64.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:45,171::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:45,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:45,252::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:45,293::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:36:45,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f521c23-c0ac-4477-aed4-4153cf34af75`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:45,294::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:36:45,315::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:45,315::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f521c23-c0ac-4477-aed4-4153cf34af75`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:45,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f521c23-c0ac-4477-aed4-4153cf34af75`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:45,316::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:45,316::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:45,316::task::993::Storage.TaskManager.Task::(_decref) Task=`1f521c23-c0ac-4477-aed4-4153cf34af75`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:36:45,327::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:36:45,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f5c1e17-a5b6-4857-81d0-3e7795dbefe3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:36:45,328::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:36:45,329::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8b2ef503-cd28-4a4c-9ba0-a93499c8039c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:36:45,330::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:36:45,330::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:36:45,331::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8b2ef503-cd28-4a4c-9ba0-a93499c8039c`::Granted request >Thread-34::DEBUG::2014-08-11 14:36:45,332::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9f5c1e17-a5b6-4857-81d0-3e7795dbefe3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:36:45,332::task::993::Storage.TaskManager.Task::(_decref) Task=`9f5c1e17-a5b6-4857-81d0-3e7795dbefe3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:36:45,357::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:45,357::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f5c1e17-a5b6-4857-81d0-3e7795dbefe3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:45,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f5c1e17-a5b6-4857-81d0-3e7795dbefe3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:36:45,358::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:36:45,358::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:36:45,358::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:36:45,359::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:36:45,359::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:36:45,359::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:36:45,359::task::993::Storage.TaskManager.Task::(_decref) Task=`9f5c1e17-a5b6-4857-81d0-3e7795dbefe3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:36:46,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:46,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131714 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:48,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:48,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136011 s, 75.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:50,161::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:50,161::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:50,161::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:50,162::__init__::232::IOProcess::(_processLogs) (5364) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:50,162::__init__::232::IOProcess::(_processLogs) DEBUG|(5364) Queuing response >Thread-16::DEBUG::2014-08-11 14:36:50,163::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:36:50,181::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000410413 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:36:50,182::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:50,183::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:50,183::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:50,183::__init__::232::IOProcess::(_processLogs) (5365) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:36:50,185::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:50,185::__init__::232::IOProcess::(_processLogs) (5366) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:50,185::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5366) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:50,186::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:50,186::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:50,187::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:50,187::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5367) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:50,187::__init__::232::IOProcess::(_processLogs) (5367) Queuing response >Thread-14::DEBUG::2014-08-11 14:36:50,188::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:36:50,188::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:36:50,188::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:36:50,189::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5368) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:36:50,189::__init__::232::IOProcess::(_processLogs) (5368) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:50,260::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:50,261::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:50,262::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:50,390::task::595::Storage.TaskManager.Task::(_updateState) Task=`67dc54a0-7db8-4fe0-a91a-e11a9e9fc858`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:50,450::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:36:50,451::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410413', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:50,451::task::1191::Storage.TaskManager.Task::(prepare) Task=`67dc54a0-7db8-4fe0-a91a-e11a9e9fc858`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000410413', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:36:50,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`67dc54a0-7db8-4fe0-a91a-e11a9e9fc858`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:50,452::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:50,452::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:50,452::task::993::Storage.TaskManager.Task::(_decref) Task=`67dc54a0-7db8-4fe0-a91a-e11a9e9fc858`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:36:50,472::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:36:50,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:50,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132494 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:52,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:52,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013134 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:54,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:54,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137333 s, 74.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:55,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:55,267::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:36:55,268::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:36:55,372::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:36:55,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a3be663-31db-4a54-8653-ef1d93d5f264`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:36:55,373::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:36:55,386::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:55,387::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a3be663-31db-4a54-8653-ef1d93d5f264`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:36:55,387::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a3be663-31db-4a54-8653-ef1d93d5f264`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:36:55,387::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:36:55,387::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:36:55,387::task::993::Storage.TaskManager.Task::(_decref) Task=`5a3be663-31db-4a54-8653-ef1d93d5f264`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:36:55,398::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:36:55,399::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e6ba6a7-47d1-491f-9b70-fef613fd8144`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:36:55,450::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:36:55,451::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ef3427f0-0f9a-480b-90d6-6bbb067a5167`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:36:55,451::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:36:55,452::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:36:55,452::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ef3427f0-0f9a-480b-90d6-6bbb067a5167`::Granted request >Thread-34::DEBUG::2014-08-11 14:36:55,453::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5e6ba6a7-47d1-491f-9b70-fef613fd8144`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:36:55,453::task::993::Storage.TaskManager.Task::(_decref) Task=`5e6ba6a7-47d1-491f-9b70-fef613fd8144`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:36:55,473::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:55,473::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e6ba6a7-47d1-491f-9b70-fef613fd8144`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:36:55,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e6ba6a7-47d1-491f-9b70-fef613fd8144`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:36:55,473::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:36:55,474::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:36:55,474::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:36:55,474::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:36:55,475::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:36:55,475::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:36:55,475::task::993::Storage.TaskManager.Task::(_decref) Task=`5e6ba6a7-47d1-491f-9b70-fef613fd8144`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:36:56,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:56,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013271 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:36:58,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:36:58,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132131 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:00,191::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:00,192::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:37:00,193::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:37:00,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:00,212::__init__::232::IOProcess::(_processLogs) (5369) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:00,212::__init__::232::IOProcess::(_processLogs) (5369) Queuing response >Thread-16::DEBUG::2014-08-11 14:37:00,213::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000886854 s, 513 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:00,214::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:00,215::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:00,215::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:00,215::__init__::232::IOProcess::(_processLogs) (5370) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:00,216::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5370) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:00,218::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:00,218::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:00,219::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:00,219::__init__::232::IOProcess::(_processLogs) (5371) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:00,219::__init__::232::IOProcess::(_processLogs) (5371) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:00,220::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:00,220::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:00,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:00,221::__init__::232::IOProcess::(_processLogs) (5372) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:00,221::__init__::232::IOProcess::(_processLogs) (5372) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:00,222::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:00,222::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:00,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:00,224::__init__::232::IOProcess::(_processLogs) (5373) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:00,224::__init__::232::IOProcess::(_processLogs) (5373) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:00,270::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:00,271::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:00,272::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:37:00,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:00,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132228 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:03,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:03,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132319 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:05,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:05,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131556 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:05,279::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:05,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:05,282::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:37:05,487::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:37:05,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2d89568-23b7-41cf-a871-933aac6301b6`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:05,488::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:37:05,501::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:05,501::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2d89568-23b7-41cf-a871-933aac6301b6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:05,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2d89568-23b7-41cf-a871-933aac6301b6`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:05,502::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:05,502::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:05,502::task::993::Storage.TaskManager.Task::(_decref) Task=`e2d89568-23b7-41cf-a871-933aac6301b6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:37:05,513::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:37:05,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`9417eb4b-dfad-4629-b3c0-d3fae75e8e80`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:37:05,514::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:37:05,514::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`644d9458-58dd-4b4e-8532-82ef716616c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:37:05,515::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:37:05,515::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:37:05,515::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`644d9458-58dd-4b4e-8532-82ef716616c7`::Granted request >Thread-34::DEBUG::2014-08-11 14:37:05,516::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9417eb4b-dfad-4629-b3c0-d3fae75e8e80`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:37:05,516::task::993::Storage.TaskManager.Task::(_decref) Task=`9417eb4b-dfad-4629-b3c0-d3fae75e8e80`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:37:05,536::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:05,536::task::1191::Storage.TaskManager.Task::(prepare) Task=`9417eb4b-dfad-4629-b3c0-d3fae75e8e80`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138710528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:05,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`9417eb4b-dfad-4629-b3c0-d3fae75e8e80`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:37:05,537::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:37:05,537::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:37:05,537::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:37:05,538::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:37:05,538::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:37:05,538::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:37:05,538::task::993::Storage.TaskManager.Task::(_decref) Task=`9417eb4b-dfad-4629-b3c0-d3fae75e8e80`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:37:05,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9dc2a9b-0714-4043-8834-61ee4724c18a`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:05,625::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:37:05,626::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000886854', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:05,626::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9dc2a9b-0714-4043-8834-61ee4724c18a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000886854', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:05,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9dc2a9b-0714-4043-8834-61ee4724c18a`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:05,626::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:05,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:05,627::task::993::Storage.TaskManager.Task::(_decref) Task=`b9dc2a9b-0714-4043-8834-61ee4724c18a`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:37:05,645::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:37:07,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:07,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133811 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:09,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:09,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131449 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:10,225::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:10,225::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:10,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:10,226::__init__::232::IOProcess::(_processLogs) (5374) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:10,227::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5374) Queuing response >Thread-16::DEBUG::2014-08-11 14:37:10,228::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:37:10,247::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418074 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:10,248::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:10,249::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:10,249::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:10,249::__init__::232::IOProcess::(_processLogs) (5375) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:10,249::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5375) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:10,251::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:10,251::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:10,251::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:10,252::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5376) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:10,252::__init__::232::IOProcess::(_processLogs) (5376) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:10,253::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:10,253::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:10,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:10,253::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5377) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:10,254::__init__::232::IOProcess::(_processLogs) (5377) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:10,254::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:10,255::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:10,255::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:10,255::__init__::232::IOProcess::(_processLogs) (5378) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:10,255::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5378) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:10,289::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:10,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:10,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:37:11,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:11,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134546 s, 76.1 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:37:11,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`b004efe3-512f-4cdc-a7fa-847b67322504`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:37:11,598::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:37:11,599::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:11,599::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:11,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:11,600::__init__::232::IOProcess::(_processLogs) (5379) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:37:11,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:11,601::__init__::232::IOProcess::(_processLogs) (5380) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:37:11,601::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5380) Queuing response >Thread-55::INFO::2014-08-11 14:37:11,602::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:37:11,602::task::1191::Storage.TaskManager.Task::(prepare) Task=`b004efe3-512f-4cdc-a7fa-847b67322504`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:37:11,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`b004efe3-512f-4cdc-a7fa-847b67322504`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:37:11,602::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:37:11,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:37:11,603::task::993::Storage.TaskManager.Task::(_decref) Task=`b004efe3-512f-4cdc-a7fa-847b67322504`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:37:13,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:13,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132648 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:15,298::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:15,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:15,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:37:15,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:15,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154587 s, 66.2 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:37:15,550::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:37:15,551::task::595::Storage.TaskManager.Task::(_updateState) Task=`283c7307-e4d2-4083-8dc9-a6458be9c294`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:15,552::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:37:15,564::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:15,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`283c7307-e4d2-4083-8dc9-a6458be9c294`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:15,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`283c7307-e4d2-4083-8dc9-a6458be9c294`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:15,565::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:15,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:15,565::task::993::Storage.TaskManager.Task::(_decref) Task=`283c7307-e4d2-4083-8dc9-a6458be9c294`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:37:15,576::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:37:15,576::task::595::Storage.TaskManager.Task::(_updateState) Task=`685ad595-9543-419a-b74a-598910c79276`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:37:15,577::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:37:15,577::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4fcc77c8-587a-48c0-a74e-83d75eec82fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:37:15,578::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:37:15,578::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:37:15,578::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4fcc77c8-587a-48c0-a74e-83d75eec82fa`::Granted request >Thread-34::DEBUG::2014-08-11 14:37:15,579::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`685ad595-9543-419a-b74a-598910c79276`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:37:15,579::task::993::Storage.TaskManager.Task::(_decref) Task=`685ad595-9543-419a-b74a-598910c79276`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:37:15,599::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138644992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:15,599::task::1191::Storage.TaskManager.Task::(prepare) Task=`685ad595-9543-419a-b74a-598910c79276`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861138644992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:15,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`685ad595-9543-419a-b74a-598910c79276`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:37:15,600::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:37:15,600::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:37:15,600::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:37:15,601::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:37:15,601::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:37:15,601::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:37:15,601::task::993::Storage.TaskManager.Task::(_decref) Task=`685ad595-9543-419a-b74a-598910c79276`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:37:17,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:17,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131508 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:19,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:19,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132285 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:20,257::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:20,258::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:20,258::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:20,258::__init__::232::IOProcess::(_processLogs) (5381) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:20,259::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5381) Queuing response >Thread-16::DEBUG::2014-08-11 14:37:20,259::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:37:20,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000470622 s, 967 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:20,280::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:20,280::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:20,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:20,280::__init__::232::IOProcess::(_processLogs) (5382) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:20,281::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5382) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:20,282::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:20,283::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:20,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:20,284::__init__::232::IOProcess::(_processLogs) (5383) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:20,284::__init__::232::IOProcess::(_processLogs) (5383) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:20,284::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:20,285::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:20,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:20,285::__init__::232::IOProcess::(_processLogs) (5384) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:20,286::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5384) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:20,287::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:20,287::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:20,288::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:20,289::__init__::232::IOProcess::(_processLogs) (5385) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:20,289::__init__::232::IOProcess::(_processLogs) (5385) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:20,307::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:20,308::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:20,309::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:37:20,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c66ab5c-1742-4ee2-98e2-cd8509c5a0a3`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:20,730::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:37:20,731::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000470622', 'lastCheck': '0.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:20,731::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c66ab5c-1742-4ee2-98e2-cd8509c5a0a3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000470622', 'lastCheck': '0.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:20,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c66ab5c-1742-4ee2-98e2-cd8509c5a0a3`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:20,731::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:20,732::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:20,732::task::993::Storage.TaskManager.Task::(_decref) Task=`5c66ab5c-1742-4ee2-98e2-cd8509c5a0a3`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:37:20,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:37:21,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:21,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134015 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:23,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:23,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131845 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:25,316::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:25,317::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:25,318::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:37:25,613::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:37:25,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4806ae5-04c5-46d0-b495-bd94f26cad68`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:25,614::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:37:25,626::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Dummy-33::DEBUG::2014-08-11 14:37:25,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-95::DEBUG::2014-08-11 14:37:25,628::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4806ae5-04c5-46d0-b495-bd94f26cad68`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:25,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4806ae5-04c5-46d0-b495-bd94f26cad68`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:25,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:25,647::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:25,647::task::993::Storage.TaskManager.Task::(_decref) Task=`a4806ae5-04c5-46d0-b495-bd94f26cad68`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:37:25,659::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:37:25,660::task::595::Storage.TaskManager.Task::(_updateState) Task=`855ecf6b-28fb-4f61-a299-be7478b43561`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:37:25,661::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:37:25,665::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed9072b2-6705-42d7-8fd3-5628a96040b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:37:25,666::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:37:25,668::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:37:25,670::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed9072b2-6705-42d7-8fd3-5628a96040b4`::Granted request >Dummy-33::DEBUG::2014-08-11 14:37:25,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224032 s, 45.7 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 14:37:25,674::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`855ecf6b-28fb-4f61-a299-be7478b43561`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:37:25,675::task::993::Storage.TaskManager.Task::(_decref) Task=`855ecf6b-28fb-4f61-a299-be7478b43561`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:37:25,726::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:25,727::task::1191::Storage.TaskManager.Task::(prepare) Task=`855ecf6b-28fb-4f61-a299-be7478b43561`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:25,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`855ecf6b-28fb-4f61-a299-be7478b43561`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:37:25,729::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:37:25,730::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:37:25,731::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:37:25,733::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:37:25,734::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:37:25,735::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:37:25,736::task::993::Storage.TaskManager.Task::(_decref) Task=`855ecf6b-28fb-4f61-a299-be7478b43561`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:37:27,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:27,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136883 s, 74.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:29,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:29,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132891 s, 77.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:37:30,289::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:37:30,290::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:30,325::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:37:30,351::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,353::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,357::__init__::232::IOProcess::(_processLogs) (5386) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:37:30,359::__init__::232::IOProcess::(_processLogs) (5386) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:30,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:30,368::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:37:30,376::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:37:30,376::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:30,377::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,377::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,377::__init__::232::IOProcess::(_processLogs) (5387) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 14:37:30,378::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:37:30,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:37:30,380::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:37:30,382::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,382::__init__::232::IOProcess::(_processLogs) (5389) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:30,382::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5389) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:30,383::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:30,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,384::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,384::__init__::232::IOProcess::(_processLogs) (5390) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:37:30,385::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 14:37:30,385::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5390) Queuing response >Thread-16::INFO::2014-08-11 14:37:30,386::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:37:30,387::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:37:30,387::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:30,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,388::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,388::__init__::232::IOProcess::(_processLogs) (5391) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:30,389::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5391) Queuing response >Thread-16::DEBUG::2014-08-11 14:37:30,390::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:37:30,409::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042263 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:30,410::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:30,410::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,411::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,411::__init__::232::IOProcess::(_processLogs) (5392) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:30,411::__init__::232::IOProcess::(_processLogs) (5392) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:30,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:30,413::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,414::__init__::232::IOProcess::(_processLogs) (5393) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:30,414::__init__::232::IOProcess::(_processLogs) (5393) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:30,414::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:30,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,416::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,417::__init__::232::IOProcess::(_processLogs) (5394) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:30,417::__init__::232::IOProcess::(_processLogs) (5394) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:30,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:30,418::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:30,419::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:30,419::__init__::232::IOProcess::(_processLogs) (5395) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:30,420::__init__::232::IOProcess::(_processLogs) (5395) Queuing response >Dummy-33::DEBUG::2014-08-11 14:37:31,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:31,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133325 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:33,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:33,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131898 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:35,379::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:35,380::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:35,451::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:37:35,753::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:37:35,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`36424e93-a95e-4d09-9f5d-e1bfba76c3a9`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:35,754::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:37:35,767::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:35,767::task::1191::Storage.TaskManager.Task::(prepare) Task=`36424e93-a95e-4d09-9f5d-e1bfba76c3a9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:35,767::task::595::Storage.TaskManager.Task::(_updateState) Task=`36424e93-a95e-4d09-9f5d-e1bfba76c3a9`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:35,768::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:35,768::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:35,768::task::993::Storage.TaskManager.Task::(_decref) Task=`36424e93-a95e-4d09-9f5d-e1bfba76c3a9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:37:35,780::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:37:35,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e621230-536b-43a5-93d9-269d2b67eebe`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:37:35,781::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:37:35,782::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`31a0513d-bed0-41ca-8fa3-8275cd95b371`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:37:35,782::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:37:35,782::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:37:35,782::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`31a0513d-bed0-41ca-8fa3-8275cd95b371`::Granted request >Thread-34::DEBUG::2014-08-11 14:37:35,785::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4e621230-536b-43a5-93d9-269d2b67eebe`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:37:35,785::task::993::Storage.TaskManager.Task::(_decref) Task=`4e621230-536b-43a5-93d9-269d2b67eebe`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:37:35,805::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:35,805::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e621230-536b-43a5-93d9-269d2b67eebe`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:35,805::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e621230-536b-43a5-93d9-269d2b67eebe`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:37:35,806::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:37:35,806::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:37:35,806::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:37:35,807::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:37:35,807::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:37:35,807::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:37:35,807::task::993::Storage.TaskManager.Task::(_decref) Task=`4e621230-536b-43a5-93d9-269d2b67eebe`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:37:35,848::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dbc390f-da96-4652-8a96-8114785d15b0`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:35,849::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:37:35,849::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042263', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:35,849::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dbc390f-da96-4652-8a96-8114785d15b0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042263', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:35,849::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dbc390f-da96-4652-8a96-8114785d15b0`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:35,850::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:35,850::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:35,850::task::993::Storage.TaskManager.Task::(_decref) Task=`9dbc390f-da96-4652-8a96-8114785d15b0`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:37:35,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:37:35,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:36,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132277 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:38,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:38,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135251 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:40,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:40,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131737 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:40,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:40,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:40,451::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:40,451::__init__::232::IOProcess::(_processLogs) (5396) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:40,451::__init__::232::IOProcess::(_processLogs) (5396) Queuing response >Thread-16::DEBUG::2014-08-11 14:37:40,452::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:40,476::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:37:40,476::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0023377 s, 195 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:40,481::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:37:40,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:40,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:40,485::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:37:40,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:40,487::__init__::232::IOProcess::(_processLogs) (5397) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:40,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5397) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:40,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:40,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:40,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:40,491::__init__::232::IOProcess::(_processLogs) (5398) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:40,491::__init__::232::IOProcess::(_processLogs) (5398) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:40,492::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:40,492::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:40,492::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:40,493::__init__::232::IOProcess::(_processLogs) (5399) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:40,493::__init__::232::IOProcess::(_processLogs) (5399) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:40,494::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:40,495::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:40,497::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:40,498::__init__::232::IOProcess::(_processLogs) (5400) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:40,499::__init__::232::IOProcess::(_processLogs) (5400) Queuing response >Dummy-33::DEBUG::2014-08-11 14:37:42,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:42,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157543 s, 65.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:44,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:44,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131971 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:45,494::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:45,495::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:45,551::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:37:45,826::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:37:45,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`118ca5e1-3131-4194-b3dc-2a16a06f2b61`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:45,827::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:37:45,840::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:45,841::task::1191::Storage.TaskManager.Task::(prepare) Task=`118ca5e1-3131-4194-b3dc-2a16a06f2b61`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:45,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`118ca5e1-3131-4194-b3dc-2a16a06f2b61`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:45,841::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:45,841::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:45,841::task::993::Storage.TaskManager.Task::(_decref) Task=`118ca5e1-3131-4194-b3dc-2a16a06f2b61`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:37:45,852::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:37:45,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`227bca09-5b9d-4f76-bfc7-f85b08742862`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:37:45,853::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:37:45,854::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3e7d2bcc-1391-4eb4-911b-631986a15964`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:37:45,854::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:37:45,854::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:37:45,854::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3e7d2bcc-1391-4eb4-911b-631986a15964`::Granted request >Thread-34::DEBUG::2014-08-11 14:37:45,855::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`227bca09-5b9d-4f76-bfc7-f85b08742862`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:37:45,855::task::993::Storage.TaskManager.Task::(_decref) Task=`227bca09-5b9d-4f76-bfc7-f85b08742862`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:37:45,875::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:45,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`227bca09-5b9d-4f76-bfc7-f85b08742862`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:45,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`227bca09-5b9d-4f76-bfc7-f85b08742862`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:37:45,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:37:45,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:37:45,877::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:37:45,877::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:37:45,877::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:37:45,877::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:37:45,878::task::993::Storage.TaskManager.Task::(_decref) Task=`227bca09-5b9d-4f76-bfc7-f85b08742862`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:37:46,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:46,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131843 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:48,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:48,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133252 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:50,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:50,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132995 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:50,497::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:50,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:50,550::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:50,551::__init__::232::IOProcess::(_processLogs) (5401) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:50,551::__init__::232::IOProcess::(_processLogs) (5401) Queuing response >Thread-16::DEBUG::2014-08-11 14:37:50,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:50,570::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:50,572::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:50,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:37:50,575::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041471 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:37:50,576::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:50,577::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:50,577::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:50,577::__init__::232::IOProcess::(_processLogs) (5402) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:37:50,578::__init__::232::IOProcess::(_processLogs) DEBUG|(5402) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:50,579::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:50,580::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:50,581::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:50,581::__init__::232::IOProcess::(_processLogs) (5403) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:50,581::__init__::232::IOProcess::(_processLogs) (5403) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:50,581::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:50,582::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:50,583::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:50,583::__init__::232::IOProcess::(_processLogs) (5404) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:50,583::__init__::232::IOProcess::(_processLogs) (5404) Queuing response >Thread-14::DEBUG::2014-08-11 14:37:50,584::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:37:50,584::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:37:50,585::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:37:50,586::__init__::232::IOProcess::(_processLogs) (5405) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:37:50,586::__init__::232::IOProcess::(_processLogs) (5405) Queuing response >Thread-95::DEBUG::2014-08-11 14:37:50,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5d10449-380a-4993-98c4-c93e4bb19736`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:50,963::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:37:50,964::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041471', 'lastCheck': '0.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:50,964::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5d10449-380a-4993-98c4-c93e4bb19736`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041471', 'lastCheck': '0.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:37:50,964::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5d10449-380a-4993-98c4-c93e4bb19736`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:50,964::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:50,965::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:50,965::task::993::Storage.TaskManager.Task::(_decref) Task=`d5d10449-380a-4993-98c4-c93e4bb19736`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:37:50,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:37:52,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:52,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133877 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:54,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:54,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133075 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:55,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:55,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:37:55,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:37:55,890::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:37:55,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`9edbb451-0c95-4be1-a043-9517e6102032`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:37:55,891::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:37:55,905::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:55,905::task::1191::Storage.TaskManager.Task::(prepare) Task=`9edbb451-0c95-4be1-a043-9517e6102032`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:37:55,905::task::595::Storage.TaskManager.Task::(_updateState) Task=`9edbb451-0c95-4be1-a043-9517e6102032`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:37:55,906::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:37:55,906::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:37:55,906::task::993::Storage.TaskManager.Task::(_decref) Task=`9edbb451-0c95-4be1-a043-9517e6102032`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:37:55,918::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:37:55,918::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3d0edb8-9570-49ac-ad71-d949708add9c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:37:55,919::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:37:55,919::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f9ab1d3a-f567-40ce-8212-d4eef384f05d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:37:55,919::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:37:55,920::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:37:55,920::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f9ab1d3a-f567-40ce-8212-d4eef384f05d`::Granted request >Thread-34::DEBUG::2014-08-11 14:37:55,921::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a3d0edb8-9570-49ac-ad71-d949708add9c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:37:55,921::task::993::Storage.TaskManager.Task::(_decref) Task=`a3d0edb8-9570-49ac-ad71-d949708add9c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:37:55,941::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:55,941::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3d0edb8-9570-49ac-ad71-d949708add9c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:37:55,941::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3d0edb8-9570-49ac-ad71-d949708add9c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:37:55,942::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:37:55,942::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:37:55,942::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:37:55,943::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:37:55,943::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:37:55,943::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:37:55,943::task::993::Storage.TaskManager.Task::(_decref) Task=`a3d0edb8-9570-49ac-ad71-d949708add9c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:37:56,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:56,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132528 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:37:58,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:37:58,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132842 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:00,587::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:00,587::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:00,591::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:00,651::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:00,652::__init__::232::IOProcess::(_processLogs) (5406) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:38:00,654::__init__::232::IOProcess::(_processLogs) (5406) Queuing response >Thread-16::DEBUG::2014-08-11 14:38:00,656::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:00,658::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:00,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 14:38:00,696::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000659447 s, 690 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:00,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:00,701::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:00,704::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:00,704::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:00,705::__init__::232::IOProcess::(_processLogs) (5407) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:38:00,705::__init__::232::IOProcess::(_processLogs) (5407) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:00,707::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:00,708::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:00,710::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:00,710::__init__::232::IOProcess::(_processLogs) (5408) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:00,711::__init__::232::IOProcess::(_processLogs) (5408) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:00,712::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:00,712::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:00,713::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:00,714::__init__::232::IOProcess::(_processLogs) (5409) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:00,715::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|(5409) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:00,717::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:00,718::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:00,720::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:00,721::__init__::232::IOProcess::(_processLogs) (5410) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:00,723::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|(5410) Queuing response >Dummy-33::DEBUG::2014-08-11 14:38:00,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028606 s, 35.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:02,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:02,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0285298 s, 35.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:04,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:04,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131569 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:05,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:05,709::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:05,751::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:38:05,955::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:38:05,956::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b31ad7a-7103-4dc0-80b0-0d4f97028382`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:05,956::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:38:05,969::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:05,970::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b31ad7a-7103-4dc0-80b0-0d4f97028382`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:05,970::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b31ad7a-7103-4dc0-80b0-0d4f97028382`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:05,970::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:05,970::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:05,970::task::993::Storage.TaskManager.Task::(_decref) Task=`2b31ad7a-7103-4dc0-80b0-0d4f97028382`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:38:05,980::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:38:05,981::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b14d6f8-4822-4398-9215-191d2eda5db9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:38:05,981::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:38:05,982::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8093608f-bcd0-4296-9c33-243508dcc777`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:38:05,982::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:38:05,983::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:38:05,983::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8093608f-bcd0-4296-9c33-243508dcc777`::Granted request >Thread-34::DEBUG::2014-08-11 14:38:05,983::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1b14d6f8-4822-4398-9215-191d2eda5db9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:38:05,984::task::993::Storage.TaskManager.Task::(_decref) Task=`1b14d6f8-4822-4398-9215-191d2eda5db9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:38:06,003::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:06,004::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b14d6f8-4822-4398-9215-191d2eda5db9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:06,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b14d6f8-4822-4398-9215-191d2eda5db9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:38:06,004::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:38:06,004::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:38:06,005::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:38:06,005::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:38:06,005::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:38:06,005::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:38:06,006::task::993::Storage.TaskManager.Task::(_decref) Task=`1b14d6f8-4822-4398-9215-191d2eda5db9`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:38:06,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ca02d24-7399-41f5-9710-2970b91ed3ea`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:06,075::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:38:06,075::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000659447', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:06,076::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ca02d24-7399-41f5-9710-2970b91ed3ea`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000659447', 'lastCheck': '5.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:06,076::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ca02d24-7399-41f5-9710-2970b91ed3ea`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:06,076::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:06,076::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:06,076::task::993::Storage.TaskManager.Task::(_decref) Task=`7ca02d24-7399-41f5-9710-2970b91ed3ea`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:38:06,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:07,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:07,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132529 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:09,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:09,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132026 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:10,725::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:10,726::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:10,751::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:38:10,752::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:38:10,752::__init__::232::IOProcess::(_processLogs) (5411) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:10,772::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:10,755::__init__::232::IOProcess::(_processLogs) (5411) Queuing response >Thread-16::DEBUG::2014-08-11 14:38:10,776::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000498698 s, 912 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:10,778::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:10,779::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:10,780::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:10,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:10,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:10,783::__init__::232::IOProcess::(_processLogs) (5412) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:38:10,786::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|(5412) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:10,788::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:10,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:10,789::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:10,789::__init__::232::IOProcess::(_processLogs) (5413) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:10,790::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|(5413) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:10,791::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:10,791::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:10,792::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:10,792::__init__::232::IOProcess::(_processLogs) (5414) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:10,793::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|(5414) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:10,793::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:10,794::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:10,794::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:10,794::__init__::232::IOProcess::(_processLogs) (5415) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:10,795::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|(5415) Queuing response >Dummy-33::DEBUG::2014-08-11 14:38:11,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:11,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131804 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:38:11,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`72ca4fd3-2a18-4e6d-ad70-e5cf81267a0d`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:38:11,707::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:38:11,708::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:11,708::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:11,708::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:11,709::__init__::232::IOProcess::(_processLogs) (5416) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:38:11,709::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|(5416) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:11,710::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:11,711::__init__::232::IOProcess::(_processLogs) (5406) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:11,711::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:11,711::__init__::232::IOProcess::(_processLogs) (5417) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:38:11,711::__init__::232::IOProcess::(_processLogs) (5417) Queuing response >Thread-55::INFO::2014-08-11 14:38:11,712::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:38:11,712::task::1191::Storage.TaskManager.Task::(prepare) Task=`72ca4fd3-2a18-4e6d-ad70-e5cf81267a0d`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:38:11,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`72ca4fd3-2a18-4e6d-ad70-e5cf81267a0d`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:38:11,713::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:38:11,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:38:11,713::task::993::Storage.TaskManager.Task::(_decref) Task=`72ca4fd3-2a18-4e6d-ad70-e5cf81267a0d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:38:13,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:13,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131474 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:15,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:15,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132115 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:15,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:15,796::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:15,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:38:16,018::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:38:16,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c240528-7dc6-4338-9207-e3600ca31116`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:16,019::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:38:16,032::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:16,032::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c240528-7dc6-4338-9207-e3600ca31116`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:16,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c240528-7dc6-4338-9207-e3600ca31116`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:16,033::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:16,033::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:16,033::task::993::Storage.TaskManager.Task::(_decref) Task=`5c240528-7dc6-4338-9207-e3600ca31116`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:38:16,043::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:38:16,044::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ee58f2e-c0be-4670-ba5b-c9b738bf67c2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:38:16,044::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:38:16,045::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0a56fbf1-1c6c-4ac3-9955-45a03e802d75`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:38:16,045::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:38:16,045::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:38:16,046::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0a56fbf1-1c6c-4ac3-9955-45a03e802d75`::Granted request >Thread-34::DEBUG::2014-08-11 14:38:16,046::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3ee58f2e-c0be-4670-ba5b-c9b738bf67c2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:38:16,046::task::993::Storage.TaskManager.Task::(_decref) Task=`3ee58f2e-c0be-4670-ba5b-c9b738bf67c2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:38:16,066::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:16,067::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ee58f2e-c0be-4670-ba5b-c9b738bf67c2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:16,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ee58f2e-c0be-4670-ba5b-c9b738bf67c2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:38:16,067::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:38:16,068::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:38:16,068::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:38:16,068::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:38:16,068::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:38:16,069::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:38:16,069::task::993::Storage.TaskManager.Task::(_decref) Task=`3ee58f2e-c0be-4670-ba5b-c9b738bf67c2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:38:17,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:17,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130679 s, 78.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:19,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:19,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133143 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:20,797::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:20,797::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:20,851::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:20,852::__init__::232::IOProcess::(_processLogs) (5418) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:38:20,852::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:38:20,853::__init__::232::IOProcess::(_processLogs) (5418) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:20,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:38:20,874::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000464536 s, 979 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:20,876::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:20,877::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:20,880::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:20,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:20,882::__init__::232::IOProcess::(_processLogs) (5419) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:38:20,883::__init__::232::IOProcess::(_processLogs) (5419) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:20,885::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:20,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:20,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:20,889::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:20,890::__init__::232::IOProcess::(_processLogs) (5420) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:20,890::__init__::232::IOProcess::(_processLogs) DEBUG|(5420) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:20,891::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:20,891::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:20,891::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:20,892::__init__::232::IOProcess::(_processLogs) (5421) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:20,892::__init__::232::IOProcess::(_processLogs) DEBUG|(5421) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:20,893::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:20,893::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:20,894::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:20,895::__init__::232::IOProcess::(_processLogs) (5422) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:20,896::__init__::232::IOProcess::(_processLogs) (5422) Queuing response >Thread-95::DEBUG::2014-08-11 14:38:21,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`73d74b21-5172-4f5e-89d2-448083b3e0f7`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:21,189::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:38:21,189::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464536', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:21,190::task::1191::Storage.TaskManager.Task::(prepare) Task=`73d74b21-5172-4f5e-89d2-448083b3e0f7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464536', 'lastCheck': '0.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:21,190::task::595::Storage.TaskManager.Task::(_updateState) Task=`73d74b21-5172-4f5e-89d2-448083b3e0f7`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:21,190::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:21,190::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:21,190::task::993::Storage.TaskManager.Task::(_decref) Task=`73d74b21-5172-4f5e-89d2-448083b3e0f7`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:38:21,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:21,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:21,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132658 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:23,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:23,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131382 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:25,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:25,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131849 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:25,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:25,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:25,951::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:38:26,083::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:38:26,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0184a84-c88b-4a5b-8cd5-9dcae2351355`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:26,084::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:38:26,096::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:26,097::task::1191::Storage.TaskManager.Task::(prepare) Task=`c0184a84-c88b-4a5b-8cd5-9dcae2351355`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:26,097::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0184a84-c88b-4a5b-8cd5-9dcae2351355`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:26,097::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:26,097::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:26,098::task::993::Storage.TaskManager.Task::(_decref) Task=`c0184a84-c88b-4a5b-8cd5-9dcae2351355`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:38:26,107::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:38:26,108::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b1f480d-fcab-4900-96bb-af16e727baff`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:38:26,108::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:38:26,109::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`25b2edaf-1cf5-411a-bd08-95082adcc914`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:38:26,109::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:38:26,110::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:38:26,110::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`25b2edaf-1cf5-411a-bd08-95082adcc914`::Granted request >Thread-34::DEBUG::2014-08-11 14:38:26,110::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4b1f480d-fcab-4900-96bb-af16e727baff`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:38:26,111::task::993::Storage.TaskManager.Task::(_decref) Task=`4b1f480d-fcab-4900-96bb-af16e727baff`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:38:26,130::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:26,131::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b1f480d-fcab-4900-96bb-af16e727baff`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:26,131::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b1f480d-fcab-4900-96bb-af16e727baff`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:38:26,131::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:38:26,131::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:38:26,132::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:38:26,132::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:38:26,132::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:38:26,133::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:38:26,133::task::993::Storage.TaskManager.Task::(_decref) Task=`4b1f480d-fcab-4900-96bb-af16e727baff`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:38:27,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:27,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131581 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:29,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:29,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132581 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:30,896::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:38:30,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:30,952::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:30,953::__init__::232::IOProcess::(_processLogs) (5423) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:38:30,953::__init__::232::IOProcess::(_processLogs) (5423) Queuing response >Thread-16::DEBUG::2014-08-11 14:38:30,954::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:30,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:38:30,974::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000499302 s, 911 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:30,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:30,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:30,978::__init__::232::IOProcess::(_processLogs) (5424) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:30,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:30,983::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:30,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:38:30,985::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:30,987::__init__::232::IOProcess::(_processLogs) (5425) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:30,988::__init__::232::IOProcess::(_processLogs) (5425) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:30,989::__init__::232::IOProcess::(_processLogs) (5426) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:30,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:30,990::__init__::232::IOProcess::(_processLogs) (5427) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:30,990::__init__::232::IOProcess::(_processLogs) (5427) Queuing response >Dummy-33::DEBUG::2014-08-11 14:38:31,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:31,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132039 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:33,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:33,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013176 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:35,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:35,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132348 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:35,994::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:35,996::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:36,053::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:38:36,144::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:38:36,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a09ffc6-4c55-4db9-8176-7049f7ca8b9e`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:36,253::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:38:36,266::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:36,266::task::1191::Storage.TaskManager.Task::(prepare) Task=`6a09ffc6-4c55-4db9-8176-7049f7ca8b9e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:36,267::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a09ffc6-4c55-4db9-8176-7049f7ca8b9e`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:36,267::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:36,267::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:36,267::task::993::Storage.TaskManager.Task::(_decref) Task=`6a09ffc6-4c55-4db9-8176-7049f7ca8b9e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:38:36,277::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:38:36,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6c87d3e-41c0-4dfd-b09d-21d782027ec6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:38:36,278::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:38:36,279::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e63473a3-c68c-4cb8-b654-ca7623617c2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:38:36,279::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:38:36,279::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:38:36,280::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e63473a3-c68c-4cb8-b654-ca7623617c2d`::Granted request >Thread-34::DEBUG::2014-08-11 14:38:36,280::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6c87d3e-41c0-4dfd-b09d-21d782027ec6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:38:36,280::task::993::Storage.TaskManager.Task::(_decref) Task=`b6c87d3e-41c0-4dfd-b09d-21d782027ec6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:38:36,300::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:36,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6c87d3e-41c0-4dfd-b09d-21d782027ec6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132091392', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:36,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6c87d3e-41c0-4dfd-b09d-21d782027ec6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:38:36,301::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:38:36,301::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:38:36,301::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:38:36,302::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:38:36,302::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:38:36,302::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:38:36,302::task::993::Storage.TaskManager.Task::(_decref) Task=`b6c87d3e-41c0-4dfd-b09d-21d782027ec6`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:38:36,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`740204ba-7473-4c72-a2fe-078064deddc0`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:36,308::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:38:36,308::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000499302', 'lastCheck': '5.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:36,308::task::1191::Storage.TaskManager.Task::(prepare) Task=`740204ba-7473-4c72-a2fe-078064deddc0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000499302', 'lastCheck': '5.3', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:36,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`740204ba-7473-4c72-a2fe-078064deddc0`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:36,309::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:36,309::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:36,309::task::993::Storage.TaskManager.Task::(_decref) Task=`740204ba-7473-4c72-a2fe-078064deddc0`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:38:36,329::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:37,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:37,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01316 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:39,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:39,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131354 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:40,992::__init__::232::IOProcess::(_processLogs) (5428) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:38:40,993::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:38:41,011::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416994 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:41,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:41,013::__init__::232::IOProcess::(_processLogs) (5429) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:38:41,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:41,015::__init__::232::IOProcess::(_processLogs) (5430) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:41,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:38:41,016::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:41,017::__init__::232::IOProcess::(_processLogs) (5431) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:41,017::__init__::232::IOProcess::(_processLogs) (5431) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:41,017::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:41,018::__init__::232::IOProcess::(_processLogs) (5432) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:41,060::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:41,061::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:41,062::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:41,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:41,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132263 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:43,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:44,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131629 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:46,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:46,070::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:46,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:46,073::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:46,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133355 s, 76.8 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:38:46,318::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:38:46,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`febdc1e9-89c5-412f-a4ea-cd5b74cc6c56`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:46,319::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:38:46,332::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:46,333::task::1191::Storage.TaskManager.Task::(prepare) Task=`febdc1e9-89c5-412f-a4ea-cd5b74cc6c56`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:46,333::task::595::Storage.TaskManager.Task::(_updateState) Task=`febdc1e9-89c5-412f-a4ea-cd5b74cc6c56`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:46,333::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:46,333::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:46,334::task::993::Storage.TaskManager.Task::(_decref) Task=`febdc1e9-89c5-412f-a4ea-cd5b74cc6c56`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:38:46,345::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:38:46,346::task::595::Storage.TaskManager.Task::(_updateState) Task=`7316242d-a1d0-4331-873f-5161940d2ac0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:38:46,346::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:38:46,347::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9353b87-c419-453a-b0a9-751edc7e4a3f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:38:46,347::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:38:46,347::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:38:46,347::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9353b87-c419-453a-b0a9-751edc7e4a3f`::Granted request >Thread-34::DEBUG::2014-08-11 14:38:46,348::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7316242d-a1d0-4331-873f-5161940d2ac0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:38:46,348::task::993::Storage.TaskManager.Task::(_decref) Task=`7316242d-a1d0-4331-873f-5161940d2ac0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:38:46,368::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132025856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:46,368::task::1191::Storage.TaskManager.Task::(prepare) Task=`7316242d-a1d0-4331-873f-5161940d2ac0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132025856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:46,368::task::595::Storage.TaskManager.Task::(_updateState) Task=`7316242d-a1d0-4331-873f-5161940d2ac0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:38:46,369::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:38:46,369::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:38:46,369::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:38:46,370::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:38:46,370::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:38:46,370::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:38:46,370::task::993::Storage.TaskManager.Task::(_decref) Task=`7316242d-a1d0-4331-873f-5161940d2ac0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:38:48,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:48,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174079 s, 58.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:50,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:50,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131762 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:51,020::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:51,020::__init__::232::IOProcess::(_processLogs) (5433) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:38:51,021::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:38:51,040::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418753 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:38:51,041::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:51,042::__init__::232::IOProcess::(_processLogs) (5434) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:38:51,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:51,043::__init__::232::IOProcess::(_processLogs) (5435) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:51,044::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:51,044::__init__::232::IOProcess::(_processLogs) (5436) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:51,044::__init__::232::IOProcess::(_processLogs) (5436) Queuing response >Thread-14::DEBUG::2014-08-11 14:38:51,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:38:51,045::__init__::232::IOProcess::(_processLogs) (5437) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:38:51,046::__init__::232::IOProcess::(_processLogs) (5437) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:51,080::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:51,081::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:51,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:38:51,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff15d5d9-5c2e-48c9-af50-d60970070fbc`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:51,449::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:38:51,449::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418753', 'lastCheck': '0.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:51,450::task::1191::Storage.TaskManager.Task::(prepare) Task=`ff15d5d9-5c2e-48c9-af50-d60970070fbc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418753', 'lastCheck': '0.4', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:38:51,450::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff15d5d9-5c2e-48c9-af50-d60970070fbc`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:51,450::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:51,450::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:51,451::task::993::Storage.TaskManager.Task::(_decref) Task=`ff15d5d9-5c2e-48c9-af50-d60970070fbc`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:38:51,469::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:52,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:52,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132197 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:38:54,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:54,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132444 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:56,160::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:56,161::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:38:56,254::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:38:56,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:56,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132684 s, 77.2 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:38:56,383::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:38:56,386::task::595::Storage.TaskManager.Task::(_updateState) Task=`61140f5e-f5f7-4aab-9b9c-ae8d5e9af5d6`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:38:56,387::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:38:56,402::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:56,403::task::1191::Storage.TaskManager.Task::(prepare) Task=`61140f5e-f5f7-4aab-9b9c-ae8d5e9af5d6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:38:56,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`61140f5e-f5f7-4aab-9b9c-ae8d5e9af5d6`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:38:56,403::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:38:56,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:38:56,404::task::993::Storage.TaskManager.Task::(_decref) Task=`61140f5e-f5f7-4aab-9b9c-ae8d5e9af5d6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:38:56,414::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:38:56,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2f2cab8-6b7e-41e1-8a16-22a476cdb8db`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:38:56,415::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:38:56,415::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`87f84930-fea9-4ed2-8dd8-3307879b4df7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:38:56,416::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:38:56,416::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:38:56,416::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`87f84930-fea9-4ed2-8dd8-3307879b4df7`::Granted request >Thread-34::DEBUG::2014-08-11 14:38:56,417::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c2f2cab8-6b7e-41e1-8a16-22a476cdb8db`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:38:56,417::task::993::Storage.TaskManager.Task::(_decref) Task=`c2f2cab8-6b7e-41e1-8a16-22a476cdb8db`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:38:56,437::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132025856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:56,437::task::1191::Storage.TaskManager.Task::(prepare) Task=`c2f2cab8-6b7e-41e1-8a16-22a476cdb8db`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132025856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:38:56,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2f2cab8-6b7e-41e1-8a16-22a476cdb8db`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:38:56,438::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:38:56,438::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:38:56,438::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:38:56,439::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:38:56,439::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:38:56,439::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:38:56,440::task::993::Storage.TaskManager.Task::(_decref) Task=`c2f2cab8-6b7e-41e1-8a16-22a476cdb8db`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:38:58,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:38:58,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132343 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:00,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:00,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135461 s, 75.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:01,048::__init__::232::IOProcess::(_processLogs) (5438) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:39:01,049::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:39:01,067::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000434427 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:01,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:01,069::__init__::232::IOProcess::(_processLogs) (5439) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:01,070::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:01,071::__init__::232::IOProcess::(_processLogs) (5440) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:01,072::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:01,072::__init__::232::IOProcess::(_processLogs) (5441) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:01,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:01,073::__init__::232::IOProcess::(_processLogs) (5442) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:01,074::__init__::232::IOProcess::(_processLogs) (5442) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:01,262::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:01,263::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:01,354::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:02,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:02,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278436 s, 36.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:04,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:04,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133252 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:06,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:06,363::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:39:06,452::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:39:06,456::task::595::Storage.TaskManager.Task::(_updateState) Task=`55e7c83b-e6f1-4a0c-a30c-942032ef7beb`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:06,458::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:06,459::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:39:06,472::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:06,472::task::1191::Storage.TaskManager.Task::(prepare) Task=`55e7c83b-e6f1-4a0c-a30c-942032ef7beb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:06,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`55e7c83b-e6f1-4a0c-a30c-942032ef7beb`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:06,473::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:06,473::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:06,473::task::993::Storage.TaskManager.Task::(_decref) Task=`55e7c83b-e6f1-4a0c-a30c-942032ef7beb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:39:06,486::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:39:06,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c14f5f9-e88b-4959-8b2e-73685d27988e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:39:06,487::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:39:06,487::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6f2f4105-286d-4239-8287-2ea963c930cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:39:06,488::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:39:06,488::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:39:06,488::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6f2f4105-286d-4239-8287-2ea963c930cb`::Granted request >Thread-34::DEBUG::2014-08-11 14:39:06,489::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2c14f5f9-e88b-4959-8b2e-73685d27988e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:39:06,489::task::993::Storage.TaskManager.Task::(_decref) Task=`2c14f5f9-e88b-4959-8b2e-73685d27988e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:39:06,509::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132025856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:06,509::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c14f5f9-e88b-4959-8b2e-73685d27988e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861132025856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:06,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c14f5f9-e88b-4959-8b2e-73685d27988e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:39:06,510::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:39:06,510::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:39:06,510::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:39:06,511::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:39:06,511::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:39:06,511::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:39:06,511::task::993::Storage.TaskManager.Task::(_decref) Task=`2c14f5f9-e88b-4959-8b2e-73685d27988e`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:39:06,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4b0939c-6735-4aba-95e7-2a7203a01189`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:06,558::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:39:06,558::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000434427', 'lastCheck': '5.5', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:06,558::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4b0939c-6735-4aba-95e7-2a7203a01189`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000434427', 'lastCheck': '5.5', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:06,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4b0939c-6735-4aba-95e7-2a7203a01189`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:06,559::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:06,559::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:06,559::task::993::Storage.TaskManager.Task::(_decref) Task=`d4b0939c-6735-4aba-95e7-2a7203a01189`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:39:06,580::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:06,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:06,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132075 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:08,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:08,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131674 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:11,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:11,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134024 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:11,075::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:11,076::__init__::232::IOProcess::(_processLogs) (5443) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:39:11,077::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:39:11,095::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427749 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:11,096::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:11,096::__init__::232::IOProcess::(_processLogs) (5444) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:11,098::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:11,098::__init__::232::IOProcess::(_processLogs) (5445) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:11,099::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5445) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:11,099::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:11,100::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:11,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:11,101::__init__::232::IOProcess::(_processLogs) (5446) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:11,101::__init__::232::IOProcess::(_processLogs) (5446) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:11,102::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:11,102::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:11,102::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:11,102::__init__::232::IOProcess::(_processLogs) (5447) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:11,103::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5447) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:11,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:11,468::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:11,554::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:39:11,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`6153568f-f73b-4714-bea3-4d4b317a09bb`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:39:11,830::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:39:11,831::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:11,831::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:11,831::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:11,832::__init__::232::IOProcess::(_processLogs) (5448) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:39:11,832::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5448) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:11,833::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:11,834::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:39:11,834::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:39:11,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:39:11,836::task::1191::Storage.TaskManager.Task::(prepare) Task=`6153568f-f73b-4714-bea3-4d4b317a09bb`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:39:11,836::__init__::232::IOProcess::(_processLogs) (5449) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:39:11,837::task::595::Storage.TaskManager.Task::(_updateState) Task=`6153568f-f73b-4714-bea3-4d4b317a09bb`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:39:11,838::__init__::232::IOProcess::(_processLogs) (5449) Queuing response >Thread-55::DEBUG::2014-08-11 14:39:11,838::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:39:11,839::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:39:11,839::task::993::Storage.TaskManager.Task::(_decref) Task=`6153568f-f73b-4714-bea3-4d4b317a09bb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:39:13,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:13,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131464 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:15,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:15,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131549 s, 77.8 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:39:16,525::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:39:16,525::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a1bbcf0-1591-4e58-98da-3777ba96f36b`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:16,553::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:16,560::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:16,563::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:16,566::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:39:16,567::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:16,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a1bbcf0-1591-4e58-98da-3777ba96f36b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:16,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a1bbcf0-1591-4e58-98da-3777ba96f36b`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:16,568::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:16,568::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:16,568::task::993::Storage.TaskManager.Task::(_decref) Task=`7a1bbcf0-1591-4e58-98da-3777ba96f36b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:39:16,578::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:39:16,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c4c7583-407b-4627-ade3-65078334a49a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:39:16,579::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:39:16,580::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ab023504-ab33-4805-92b3-1b04c8d3eade`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:39:16,580::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:39:16,581::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:39:16,581::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ab023504-ab33-4805-92b3-1b04c8d3eade`::Granted request >Thread-34::DEBUG::2014-08-11 14:39:16,582::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4c4c7583-407b-4627-ade3-65078334a49a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:39:16,582::task::993::Storage.TaskManager.Task::(_decref) Task=`4c4c7583-407b-4627-ade3-65078334a49a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:39:16,602::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:16,602::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c4c7583-407b-4627-ade3-65078334a49a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:16,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c4c7583-407b-4627-ade3-65078334a49a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:39:16,602::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:39:16,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:39:16,603::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:39:16,603::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:39:16,603::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:39:16,604::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:39:16,604::task::993::Storage.TaskManager.Task::(_decref) Task=`4c4c7583-407b-4627-ade3-65078334a49a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:39:17,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:17,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013163 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:19,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:19,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132882 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:21,104::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:21,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:21,106::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:39:21,107::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:39:21,107::__init__::232::IOProcess::(_processLogs) (5450) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:21,125::__init__::232::IOProcess::(_processLogs) (5450) Queuing response >Thread-16::DEBUG::2014-08-11 14:39:21,126::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000557403 s, 816 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:21,127::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:21,128::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:21,128::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:21,128::__init__::232::IOProcess::(_processLogs) (5451) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:21,129::__init__::232::IOProcess::(_processLogs) DEBUG|(5451) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:21,130::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:21,131::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:21,132::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:21,132::__init__::232::IOProcess::(_processLogs) (5452) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:21,132::__init__::232::IOProcess::(_processLogs) (5452) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:21,132::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:21,133::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:21,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:21,133::__init__::232::IOProcess::(_processLogs) (5453) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:21,134::__init__::232::IOProcess::(_processLogs) DEBUG|(5453) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:21,135::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:21,135::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:21,136::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:21,137::__init__::232::IOProcess::(_processLogs) (5454) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:21,137::__init__::232::IOProcess::(_processLogs) (5454) Queuing response >Dummy-33::DEBUG::2014-08-11 14:39:21,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:21,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131946 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:21,573::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:21,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:21,654::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:39:21,671::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e037ccc-20b2-4137-93ee-f54f8fa536a8`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:21,671::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:39:21,672::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000557403', 'lastCheck': '0.5', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:21,672::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e037ccc-20b2-4137-93ee-f54f8fa536a8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000557403', 'lastCheck': '0.5', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:21,672::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e037ccc-20b2-4137-93ee-f54f8fa536a8`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:21,672::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:21,673::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:21,673::task::993::Storage.TaskManager.Task::(_decref) Task=`7e037ccc-20b2-4137-93ee-f54f8fa536a8`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:39:21,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:23,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:23,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171401 s, 59.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:25,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:25,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132666 s, 77.2 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:39:26,616::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:39:26,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`50afca52-dfcd-45e3-a5be-1bfc617edcc9`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:26,617::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:39:26,630::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:26,630::task::1191::Storage.TaskManager.Task::(prepare) Task=`50afca52-dfcd-45e3-a5be-1bfc617edcc9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:26,630::task::595::Storage.TaskManager.Task::(_updateState) Task=`50afca52-dfcd-45e3-a5be-1bfc617edcc9`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:26,631::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:26,631::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:26,631::task::993::Storage.TaskManager.Task::(_decref) Task=`50afca52-dfcd-45e3-a5be-1bfc617edcc9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:39:26,641::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:39:26,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`170396f9-722f-4423-a86b-1d8a501df31e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:39:26,642::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:39:26,643::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`715f9efd-03ea-41b1-adbd-dbaaf8733cc6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:39:26,643::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:39:26,645::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:39:26,646::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`715f9efd-03ea-41b1-adbd-dbaaf8733cc6`::Granted request >Thread-34::DEBUG::2014-08-11 14:39:26,646::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`170396f9-722f-4423-a86b-1d8a501df31e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:39:26,646::task::993::Storage.TaskManager.Task::(_decref) Task=`170396f9-722f-4423-a86b-1d8a501df31e`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:26,660::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:26,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:26,665::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:39:26,667::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:26,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`170396f9-722f-4423-a86b-1d8a501df31e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:26,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`170396f9-722f-4423-a86b-1d8a501df31e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:39:26,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:39:26,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:39:26,668::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:39:26,668::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:39:26,669::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:39:26,669::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:39:26,669::task::993::Storage.TaskManager.Task::(_decref) Task=`170396f9-722f-4423-a86b-1d8a501df31e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:39:27,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:27,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131827 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:29,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:29,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131711 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:31,138::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:31,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:31,139::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:39:31,140::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:39:31,141::__init__::232::IOProcess::(_processLogs) (5455) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:31,159::__init__::232::IOProcess::(_processLogs) (5455) Queuing response >Thread-16::DEBUG::2014-08-11 14:39:31,160::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422661 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:31,161::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:31,162::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:31,163::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:31,164::__init__::232::IOProcess::(_processLogs) (5456) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:31,164::__init__::232::IOProcess::(_processLogs) (5456) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:31,164::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:31,165::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:31,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:31,166::__init__::232::IOProcess::(_processLogs) (5457) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:31,166::__init__::232::IOProcess::(_processLogs) (5457) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:31,166::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:31,167::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:31,168::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:31,168::__init__::232::IOProcess::(_processLogs) (5458) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:31,168::__init__::232::IOProcess::(_processLogs) (5458) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:31,169::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:31,169::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:31,170::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:31,171::__init__::232::IOProcess::(_processLogs) (5459) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:31,171::__init__::232::IOProcess::(_processLogs) (5459) Queuing response >Dummy-33::DEBUG::2014-08-11 14:39:31,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:31,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133523 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:31,670::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:31,671::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:31,755::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:33,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:33,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131927 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:35,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:35,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132791 s, 77.1 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:39:36,681::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:39:36,682::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa1570e7-c00c-46bd-883f-a6742a783f26`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:36,754::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:36,763::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:36,765::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:36,766::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:39:36,767::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:36,768::task::1191::Storage.TaskManager.Task::(prepare) Task=`aa1570e7-c00c-46bd-883f-a6742a783f26`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:36,768::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa1570e7-c00c-46bd-883f-a6742a783f26`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:36,769::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:36,769::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:36,769::task::993::Storage.TaskManager.Task::(_decref) Task=`aa1570e7-c00c-46bd-883f-a6742a783f26`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:39:36,780::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:39:36,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd3e68a1-9697-4105-948f-9b2cb6d01862`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:39:36,781::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:39:36,781::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f15a667f-9320-4577-ac5e-3a44ecc5a829`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:39:36,781::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:39:36,782::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:39:36,782::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f15a667f-9320-4577-ac5e-3a44ecc5a829`::Granted request >Thread-34::DEBUG::2014-08-11 14:39:36,783::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cd3e68a1-9697-4105-948f-9b2cb6d01862`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:39:36,783::task::993::Storage.TaskManager.Task::(_decref) Task=`cd3e68a1-9697-4105-948f-9b2cb6d01862`::ref 1 aborting False >Thread-95::DEBUG::2014-08-11 14:39:36,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ce603fa-71be-40d8-9001-4a975d2da680`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:36,800::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:39:36,800::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422661', 'lastCheck': '5.6', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:36,801::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ce603fa-71be-40d8-9001-4a975d2da680`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422661', 'lastCheck': '5.6', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:36,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ce603fa-71be-40d8-9001-4a975d2da680`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:36,801::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:36,801::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:36,802::task::993::Storage.TaskManager.Task::(_decref) Task=`0ce603fa-71be-40d8-9001-4a975d2da680`::ref 0 aborting False >Thread-34::INFO::2014-08-11 14:39:36,804::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:36,806::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd3e68a1-9697-4105-948f-9b2cb6d01862`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:36,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd3e68a1-9697-4105-948f-9b2cb6d01862`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:39:36,808::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:39:36,809::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:39:36,811::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:39:36,813::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:39:36,814::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:39:36,815::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:39:36,817::task::993::Storage.TaskManager.Task::(_decref) Task=`cd3e68a1-9697-4105-948f-9b2cb6d01862`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:39:36,846::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:37,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:37,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132068 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:39,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:39,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134349 s, 76.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:41,171::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:41,172::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:41,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:41,172::__init__::232::IOProcess::(_processLogs) (5460) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:41,173::__init__::232::IOProcess::(_processLogs) DEBUG|(5460) Queuing response >Thread-16::DEBUG::2014-08-11 14:39:41,174::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:39:41,192::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417466 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:41,194::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:41,194::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:41,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:41,195::__init__::232::IOProcess::(_processLogs) (5461) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:41,195::__init__::232::IOProcess::(_processLogs) DEBUG|(5461) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:41,197::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:41,198::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:41,199::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:41,199::__init__::232::IOProcess::(_processLogs) (5462) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:41,199::__init__::232::IOProcess::(_processLogs) (5462) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:41,200::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:41,200::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:41,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:41,201::__init__::232::IOProcess::(_processLogs) (5463) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:41,201::__init__::232::IOProcess::(_processLogs) (5463) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:41,202::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:41,202::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:41,203::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:41,204::__init__::232::IOProcess::(_processLogs) (5464) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:41,204::__init__::232::IOProcess::(_processLogs) (5464) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:41,774::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:41,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:41,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:41,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:41,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132621 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:43,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:43,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131758 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:45,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:46,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131604 s, 77.8 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:39:46,846::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:39:46,855::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebd553c8-99d0-4314-845c-fab48bcbdd49`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:46,855::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:39:46,868::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:46,869::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebd553c8-99d0-4314-845c-fab48bcbdd49`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:46,869::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebd553c8-99d0-4314-845c-fab48bcbdd49`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:46,869::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:46,869::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:46,870::task::993::Storage.TaskManager.Task::(_decref) Task=`ebd553c8-99d0-4314-845c-fab48bcbdd49`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:46,882::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:46,883::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:46,884::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:39:46,888::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:39:46,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`d221ee81-ced4-48ab-a0a2-8e3d43b4e025`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:39:46,889::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:39:46,890::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`74650165-995b-443f-9bf6-de0697233cc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:39:46,890::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:39:46,891::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:39:46,891::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`74650165-995b-443f-9bf6-de0697233cc0`::Granted request >Thread-34::DEBUG::2014-08-11 14:39:46,891::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d221ee81-ced4-48ab-a0a2-8e3d43b4e025`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:39:46,892::task::993::Storage.TaskManager.Task::(_decref) Task=`d221ee81-ced4-48ab-a0a2-8e3d43b4e025`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:39:46,911::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:46,912::task::1191::Storage.TaskManager.Task::(prepare) Task=`d221ee81-ced4-48ab-a0a2-8e3d43b4e025`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:46,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`d221ee81-ced4-48ab-a0a2-8e3d43b4e025`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:39:46,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:39:46,913::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:39:46,915::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:39:46,917::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:39:46,918::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:39:46,919::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:39:46,922::task::993::Storage.TaskManager.Task::(_decref) Task=`d221ee81-ced4-48ab-a0a2-8e3d43b4e025`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:39:48,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:48,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132028 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:50,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:50,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133026 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:51,205::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:51,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:39:51,207::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:39:51,207::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:51,226::__init__::232::IOProcess::(_processLogs) (5465) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:51,226::__init__::232::IOProcess::(_processLogs) (5465) Queuing response >Thread-16::DEBUG::2014-08-11 14:39:51,227::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000490437 s, 928 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:39:51,228::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:51,228::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:51,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:51,229::__init__::232::IOProcess::(_processLogs) (5466) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:39:51,230::__init__::232::IOProcess::(_processLogs) DEBUG|(5466) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:51,231::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:51,232::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:51,232::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:51,233::__init__::232::IOProcess::(_processLogs) (5467) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:51,233::__init__::232::IOProcess::(_processLogs) (5467) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:51,233::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:51,234::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:51,234::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:51,235::__init__::232::IOProcess::(_processLogs) (5468) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:51,235::__init__::232::IOProcess::(_processLogs) (5468) Queuing response >Thread-14::DEBUG::2014-08-11 14:39:51,235::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:39:51,236::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:39:51,236::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:39:51,237::__init__::232::IOProcess::(_processLogs) (5469) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:39:51,238::__init__::232::IOProcess::(_processLogs) (5469) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:51,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:51,892::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:39:51,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba6170d1-bddc-4943-92de-bd53eafc305b`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:51,956::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:51,958::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:39:51,958::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000490437', 'lastCheck': '0.7', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:51,960::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba6170d1-bddc-4943-92de-bd53eafc305b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000490437', 'lastCheck': '0.7', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:39:51,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba6170d1-bddc-4943-92de-bd53eafc305b`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:51,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:51,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:51,963::task::993::Storage.TaskManager.Task::(_decref) Task=`ba6170d1-bddc-4943-92de-bd53eafc305b`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:39:51,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:39:52,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:52,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131793 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:54,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:54,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132686 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:39:56,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:56,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132431 s, 77.3 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:39:56,936::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:39:56,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c3378be-381c-4e8a-b036-60600f3e3122`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:39:56,955::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:56,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:56,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:39:56,970::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:39:56,981::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:56,982::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c3378be-381c-4e8a-b036-60600f3e3122`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:39:56,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c3378be-381c-4e8a-b036-60600f3e3122`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:39:56,984::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:39:56,985::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:39:56,986::task::993::Storage.TaskManager.Task::(_decref) Task=`1c3378be-381c-4e8a-b036-60600f3e3122`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:39:56,998::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:39:56,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`22262a51-733e-4a89-846f-18b206e38f29`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:39:56,999::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:39:57,000::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fd71b2de-a96f-418a-9f9a-3d82831ca89c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:39:57,000::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:39:57,001::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:39:57,001::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fd71b2de-a96f-418a-9f9a-3d82831ca89c`::Granted request >Thread-34::DEBUG::2014-08-11 14:39:57,002::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`22262a51-733e-4a89-846f-18b206e38f29`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:39:57,002::task::993::Storage.TaskManager.Task::(_decref) Task=`22262a51-733e-4a89-846f-18b206e38f29`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:39:57,022::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:57,022::task::1191::Storage.TaskManager.Task::(prepare) Task=`22262a51-733e-4a89-846f-18b206e38f29`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131894784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:39:57,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`22262a51-733e-4a89-846f-18b206e38f29`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:39:57,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:39:57,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:39:57,023::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:39:57,024::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:39:57,024::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:39:57,024::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:39:57,024::task::993::Storage.TaskManager.Task::(_decref) Task=`22262a51-733e-4a89-846f-18b206e38f29`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:39:58,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:39:58,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131541 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:00,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:00,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134602 s, 76.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:01,238::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:01,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:40:01,240::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:40:01,241::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:01,259::__init__::232::IOProcess::(_processLogs) (5470) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:01,260::__init__::232::IOProcess::(_processLogs) (5470) Queuing response >Thread-16::DEBUG::2014-08-11 14:40:01,261::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000450099 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:01,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:01,263::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:01,263::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:01,263::__init__::232::IOProcess::(_processLogs) (5471) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:01,264::__init__::232::IOProcess::(_processLogs) (5471) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:01,265::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:01,266::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:01,266::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:01,266::__init__::232::IOProcess::(_processLogs) (5472) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:01,267::__init__::232::IOProcess::(_processLogs) (5472) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:01,267::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:01,268::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:01,268::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:01,268::__init__::232::IOProcess::(_processLogs) (5473) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:01,269::__init__::232::IOProcess::(_processLogs) (5473) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:01,269::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:01,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:01,270::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:01,271::__init__::232::IOProcess::(_processLogs) (5474) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:01,272::__init__::232::IOProcess::(_processLogs) (5474) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:01,979::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:01,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:01,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:40:02,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:02,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274154 s, 37.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:04,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:04,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132074 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:06,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:06,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133542 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:06,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:07,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:07,001::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:40:07,038::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:40:07,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`1502ffb2-39cc-45fc-aa57-003cbdfbde2a`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:07,039::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:40:07,052::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:07,053::task::1191::Storage.TaskManager.Task::(prepare) Task=`1502ffb2-39cc-45fc-aa57-003cbdfbde2a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:07,053::task::595::Storage.TaskManager.Task::(_updateState) Task=`1502ffb2-39cc-45fc-aa57-003cbdfbde2a`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:07,053::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:07,054::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:07,054::task::993::Storage.TaskManager.Task::(_decref) Task=`1502ffb2-39cc-45fc-aa57-003cbdfbde2a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:40:07,064::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:40:07,065::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b05f18d-563c-4644-89f0-489810c22f37`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:40:07,065::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:40:07,066::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d001de27-6e23-4170-a619-e70599296438`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:40:07,066::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:40:07,066::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:40:07,066::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d001de27-6e23-4170-a619-e70599296438`::Granted request >Thread-34::DEBUG::2014-08-11 14:40:07,067::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6b05f18d-563c-4644-89f0-489810c22f37`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:40:07,067::task::993::Storage.TaskManager.Task::(_decref) Task=`6b05f18d-563c-4644-89f0-489810c22f37`::ref 1 aborting False >Thread-95::DEBUG::2014-08-11 14:40:07,077::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a5f7856-0e6d-4816-aa5a-31303467a4bc`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:07,079::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:40:07,081::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000450099', 'lastCheck': '5.8', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:07,083::task::1191::Storage.TaskManager.Task::(prepare) Task=`6a5f7856-0e6d-4816-aa5a-31303467a4bc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000450099', 'lastCheck': '5.8', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:07,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a5f7856-0e6d-4816-aa5a-31303467a4bc`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:07,086::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:07,088::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:07,089::task::993::Storage.TaskManager.Task::(_decref) Task=`6a5f7856-0e6d-4816-aa5a-31303467a4bc`::ref 0 aborting False >Thread-34::INFO::2014-08-11 14:40:07,102::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131829248', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:07,103::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b05f18d-563c-4644-89f0-489810c22f37`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861131829248', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:07,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b05f18d-563c-4644-89f0-489810c22f37`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:40:07,103::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:40:07,103::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:40:07,104::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:40:07,104::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:40:07,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:40:07,104::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:40:07,105::task::993::Storage.TaskManager.Task::(_decref) Task=`6b05f18d-563c-4644-89f0-489810c22f37`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:40:07,110::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:40:08,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:08,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133239 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:10,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:10,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132555 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:11,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:11,273::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:11,273::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:11,273::__init__::232::IOProcess::(_processLogs) (5475) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:11,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5475) Queuing response >Thread-16::DEBUG::2014-08-11 14:40:11,275::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:40:11,295::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000441249 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:11,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:11,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:11,297::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:11,297::__init__::232::IOProcess::(_processLogs) (5476) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:11,297::__init__::232::IOProcess::(_processLogs) (5476) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:11,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:11,299::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:11,299::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:11,299::__init__::232::IOProcess::(_processLogs) (5477) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:11,300::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5477) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:11,300::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:11,301::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:11,302::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:11,302::__init__::232::IOProcess::(_processLogs) (5478) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:11,302::__init__::232::IOProcess::(_processLogs) (5478) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:11,302::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:11,303::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:11,303::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:11,304::__init__::232::IOProcess::(_processLogs) (5479) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:11,305::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5479) Queuing response >Thread-55::DEBUG::2014-08-11 14:40:11,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5afcb74-3bb3-495b-afc0-ce2120ea4671`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:40:11,974::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:12,008::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:40:12,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:12,057::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:12,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:12,060::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:40:12,061::__init__::232::IOProcess::(_processLogs) (5480) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:40:12,064::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5480) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:12,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:40:12,066::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:12,068::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:12,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:12,069::__init__::232::IOProcess::(_processLogs) (5481) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:40:12,070::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:40:12,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5481) Queuing response >Thread-55::DEBUG::2014-08-11 14:40:12,071::task::1191::Storage.TaskManager.Task::(prepare) Task=`e5afcb74-3bb3-495b-afc0-ce2120ea4671`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:40:12,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5afcb74-3bb3-495b-afc0-ce2120ea4671`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:40:12,072::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:40:12,072::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:40:12,073::task::993::Storage.TaskManager.Task::(_decref) Task=`e5afcb74-3bb3-495b-afc0-ce2120ea4671`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:40:12,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:12,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139227 s, 73.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:14,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:15,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133851 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:17,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:17,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148919 s, 68.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:17,077::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:17,083::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:17,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:40:17,116::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:40:17,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`708cdeb6-643d-4e42-a7b5-a5f5e5258360`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:17,117::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:40:17,130::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:17,130::task::1191::Storage.TaskManager.Task::(prepare) Task=`708cdeb6-643d-4e42-a7b5-a5f5e5258360`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:17,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`708cdeb6-643d-4e42-a7b5-a5f5e5258360`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:17,130::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:17,131::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:17,131::task::993::Storage.TaskManager.Task::(_decref) Task=`708cdeb6-643d-4e42-a7b5-a5f5e5258360`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:40:17,141::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:40:17,142::task::595::Storage.TaskManager.Task::(_updateState) Task=`03a2815c-abdf-44ed-9bbc-2f59c05c2e08`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:40:17,142::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:40:17,143::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`06607607-e041-4134-b9b7-277511a4ef5d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:40:17,143::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:40:17,143::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:40:17,144::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`06607607-e041-4134-b9b7-277511a4ef5d`::Granted request >Thread-34::DEBUG::2014-08-11 14:40:17,144::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`03a2815c-abdf-44ed-9bbc-2f59c05c2e08`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:40:17,144::task::993::Storage.TaskManager.Task::(_decref) Task=`03a2815c-abdf-44ed-9bbc-2f59c05c2e08`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:40:17,164::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860870930432', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:17,165::task::1191::Storage.TaskManager.Task::(prepare) Task=`03a2815c-abdf-44ed-9bbc-2f59c05c2e08`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860870930432', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:17,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`03a2815c-abdf-44ed-9bbc-2f59c05c2e08`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:40:17,165::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:40:17,165::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:40:17,166::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:40:17,166::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:40:17,166::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:40:17,166::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:40:17,167::task::993::Storage.TaskManager.Task::(_decref) Task=`03a2815c-abdf-44ed-9bbc-2f59c05c2e08`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:40:19,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:19,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134043 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:21,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:21,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132262 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:21,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:21,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:21,307::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:21,307::__init__::232::IOProcess::(_processLogs) (5482) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:21,308::__init__::232::IOProcess::(_processLogs) (5482) Queuing response >Thread-16::DEBUG::2014-08-11 14:40:21,308::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:40:21,327::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419852 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:21,328::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:21,328::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:21,329::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:21,329::__init__::232::IOProcess::(_processLogs) (5483) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:21,329::__init__::232::IOProcess::(_processLogs) (5483) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:21,330::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:21,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:21,331::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:21,332::__init__::232::IOProcess::(_processLogs) (5484) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:21,332::__init__::232::IOProcess::(_processLogs) (5484) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:21,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:21,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:21,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:21,333::__init__::232::IOProcess::(_processLogs) (5485) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:21,334::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5485) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:21,334::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:21,335::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:21,335::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:21,337::__init__::232::IOProcess::(_processLogs) (5486) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:21,337::__init__::232::IOProcess::(_processLogs) (5486) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:22,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:22,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:22,155::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:40:22,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`90db31d8-1465-460c-81ec-c0924f151911`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:22,199::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:40:22,200::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419852', 'lastCheck': '0.9', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:22,200::task::1191::Storage.TaskManager.Task::(prepare) Task=`90db31d8-1465-460c-81ec-c0924f151911`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419852', 'lastCheck': '0.9', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:22,200::task::595::Storage.TaskManager.Task::(_updateState) Task=`90db31d8-1465-460c-81ec-c0924f151911`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:22,200::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:22,200::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:22,201::task::993::Storage.TaskManager.Task::(_decref) Task=`90db31d8-1465-460c-81ec-c0924f151911`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:40:22,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:40:23,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:23,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131813 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:25,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:25,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179909 s, 56.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:27,162::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:27,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:27,165::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:40:27,179::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:40:27,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fded8f1-9b73-4263-b8b1-bcc2874b875f`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:27,180::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:40:27,193::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:27,193::task::1191::Storage.TaskManager.Task::(prepare) Task=`6fded8f1-9b73-4263-b8b1-bcc2874b875f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:27,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fded8f1-9b73-4263-b8b1-bcc2874b875f`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:27,194::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:27,194::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:27,194::task::993::Storage.TaskManager.Task::(_decref) Task=`6fded8f1-9b73-4263-b8b1-bcc2874b875f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:40:27,205::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:40:27,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`65b247c6-c3b5-4a39-8164-ab36df773f9d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:40:27,206::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:40:27,206::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f06d4d0e-6581-4905-8f96-6dc53bec4ada`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:40:27,207::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:40:27,207::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:40:27,207::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f06d4d0e-6581-4905-8f96-6dc53bec4ada`::Granted request >Thread-34::DEBUG::2014-08-11 14:40:27,208::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`65b247c6-c3b5-4a39-8164-ab36df773f9d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:40:27,208::task::993::Storage.TaskManager.Task::(_decref) Task=`65b247c6-c3b5-4a39-8164-ab36df773f9d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:40:27,228::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860860968960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:27,228::task::1191::Storage.TaskManager.Task::(prepare) Task=`65b247c6-c3b5-4a39-8164-ab36df773f9d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860860968960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:27,229::task::595::Storage.TaskManager.Task::(_updateState) Task=`65b247c6-c3b5-4a39-8164-ab36df773f9d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:40:27,229::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:40:27,229::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:40:27,229::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:40:27,230::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:40:27,230::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:40:27,230::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:40:27,230::task::993::Storage.TaskManager.Task::(_decref) Task=`65b247c6-c3b5-4a39-8164-ab36df773f9d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:40:27,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:27,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132563 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:29,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:29,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133598 s, 76.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:31,337::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:31,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:31,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:31,340::__init__::232::IOProcess::(_processLogs) (5487) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:40:31,340::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:40:31,360::__init__::232::IOProcess::(_processLogs) (5487) Queuing response >Thread-16::DEBUG::2014-08-11 14:40:31,363::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432084 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:31,365::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:31,365::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:31,365::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:31,365::__init__::232::IOProcess::(_processLogs) (5488) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:31,366::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5488) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:31,367::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:31,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:31,369::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:31,369::__init__::232::IOProcess::(_processLogs) (5489) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:31,370::__init__::232::IOProcess::(_processLogs) (5489) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:31,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:31,371::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:31,371::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:31,371::__init__::232::IOProcess::(_processLogs) (5490) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:31,372::__init__::232::IOProcess::(_processLogs) (5490) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:31,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:31,373::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:31,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:31,374::__init__::232::IOProcess::(_processLogs) (5491) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:31,374::__init__::232::IOProcess::(_processLogs) (5491) Queuing response >Dummy-33::DEBUG::2014-08-11 14:40:31,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:31,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136565 s, 75.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:32,172::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:32,174::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:32,259::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:40:33,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:33,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131514 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:35,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:35,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132161 s, 77.5 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:40:37,243::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:40:37,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`68fdf08b-355e-446f-b665-889412aeec97`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:37,254::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:40:37,267::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:37,268::task::1191::Storage.TaskManager.Task::(prepare) Task=`68fdf08b-355e-446f-b665-889412aeec97`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:37,268::task::595::Storage.TaskManager.Task::(_updateState) Task=`68fdf08b-355e-446f-b665-889412aeec97`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:37,268::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:37,269::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:37,271::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:40:37,271::task::993::Storage.TaskManager.Task::(_decref) Task=`68fdf08b-355e-446f-b665-889412aeec97`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:37,275::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:37,276::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:40:37,285::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:40:37,285::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea322bc4-aa96-4bc6-bc90-751ac27a6d3e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:40:37,286::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:40:37,286::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30f6f6d6-9f32-42d3-9aba-22928886fd5a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:40:37,287::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:40:37,287::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:40:37,287::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30f6f6d6-9f32-42d3-9aba-22928886fd5a`::Granted request >Thread-34::DEBUG::2014-08-11 14:40:37,288::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ea322bc4-aa96-4bc6-bc90-751ac27a6d3e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:40:37,288::task::993::Storage.TaskManager.Task::(_decref) Task=`ea322bc4-aa96-4bc6-bc90-751ac27a6d3e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:40:37,308::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129535488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:37,308::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea322bc4-aa96-4bc6-bc90-751ac27a6d3e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129535488', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:37,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea322bc4-aa96-4bc6-bc90-751ac27a6d3e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:40:37,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:40:37,309::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:40:37,309::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:40:37,309::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:40:37,310::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:40:37,310::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:40:37,310::task::993::Storage.TaskManager.Task::(_decref) Task=`ea322bc4-aa96-4bc6-bc90-751ac27a6d3e`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:40:37,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a49afc0-9540-4def-a853-536628f5f6fa`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:37,388::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:40:37,388::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432084', 'lastCheck': '6.0', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:37,388::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a49afc0-9540-4def-a853-536628f5f6fa`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432084', 'lastCheck': '6.0', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:37,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a49afc0-9540-4def-a853-536628f5f6fa`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:37,389::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:37,389::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:37,389::task::993::Storage.TaskManager.Task::(_decref) Task=`1a49afc0-9540-4def-a853-536628f5f6fa`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:40:37,410::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:40:37,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:37,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131732 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:39,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:39,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132371 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:41,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:41,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:41,376::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:41,376::__init__::232::IOProcess::(_processLogs) (5492) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:41,376::__init__::232::IOProcess::(_processLogs) (5492) Queuing response >Thread-16::DEBUG::2014-08-11 14:40:41,377::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:40:41,395::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423181 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:41,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:41,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:41,397::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:41,397::__init__::232::IOProcess::(_processLogs) (5493) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:41,398::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(5493) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:41,400::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:41,400::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:41,401::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:41,401::__init__::232::IOProcess::(_processLogs) (5494) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:41,401::__init__::232::IOProcess::(_processLogs) (5494) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:41,402::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:41,402::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:41,403::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:41,403::__init__::232::IOProcess::(_processLogs) (5495) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:41,403::__init__::232::IOProcess::(_processLogs) (5495) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:41,404::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:41,404::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:41,405::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:41,405::__init__::232::IOProcess::(_processLogs) (5496) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:41,406::__init__::232::IOProcess::(_processLogs) (5496) Queuing response >Dummy-33::DEBUG::2014-08-11 14:40:41,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:41,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131724 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:42,283::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:42,284::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:42,355::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:40:43,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:43,812::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132053 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:45,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:45,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131334 s, 78.0 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:40:47,323::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:40:47,324::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac2741cb-a17d-48bb-8e6d-e6af5073cc53`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:47,354::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:47,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:47,363::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:47,366::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:40:47,367::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:47,368::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac2741cb-a17d-48bb-8e6d-e6af5073cc53`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:47,368::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac2741cb-a17d-48bb-8e6d-e6af5073cc53`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:47,368::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:47,368::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:47,368::task::993::Storage.TaskManager.Task::(_decref) Task=`ac2741cb-a17d-48bb-8e6d-e6af5073cc53`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:40:47,382::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:40:47,383::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c81cb84-3086-46d9-9e20-94b51768f5bf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:40:47,383::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:40:47,384::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0253cb8f-58f8-4c11-81ab-9ae7f1f9acd1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:40:47,384::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:40:47,384::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:40:47,384::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0253cb8f-58f8-4c11-81ab-9ae7f1f9acd1`::Granted request >Thread-34::DEBUG::2014-08-11 14:40:47,385::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9c81cb84-3086-46d9-9e20-94b51768f5bf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:40:47,385::task::993::Storage.TaskManager.Task::(_decref) Task=`9c81cb84-3086-46d9-9e20-94b51768f5bf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:40:47,405::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:47,405::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c81cb84-3086-46d9-9e20-94b51768f5bf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:47,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c81cb84-3086-46d9-9e20-94b51768f5bf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:40:47,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:40:47,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:40:47,406::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:40:47,407::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:40:47,407::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:40:47,407::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:40:47,407::task::993::Storage.TaskManager.Task::(_decref) Task=`9c81cb84-3086-46d9-9e20-94b51768f5bf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:40:47,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:47,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131446 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:49,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:49,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131983 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:51,406::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:51,407::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:51,408::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:40:51,409::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:40:51,409::__init__::232::IOProcess::(_processLogs) (5497) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:51,427::__init__::232::IOProcess::(_processLogs) (5497) Queuing response >Thread-16::DEBUG::2014-08-11 14:40:51,429::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422533 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:40:51,430::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:51,430::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:51,430::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:51,430::__init__::232::IOProcess::(_processLogs) (5498) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:40:51,431::__init__::232::IOProcess::(_processLogs) DEBUG|(5498) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:51,433::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:51,433::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:51,434::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:51,434::__init__::232::IOProcess::(_processLogs) (5499) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:51,434::__init__::232::IOProcess::(_processLogs) (5499) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:51,435::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:51,435::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:51,436::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:51,436::__init__::232::IOProcess::(_processLogs) (5500) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:51,437::__init__::232::IOProcess::(_processLogs) (5500) Queuing response >Thread-14::DEBUG::2014-08-11 14:40:51,437::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:40:51,437::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:40:51,438::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:40:51,439::__init__::232::IOProcess::(_processLogs) (5501) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:40:51,439::__init__::232::IOProcess::(_processLogs) (5501) Queuing response >Dummy-33::DEBUG::2014-08-11 14:40:51,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:52,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132121 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:52,373::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:52,374::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:52,455::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:40:52,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9d9ed0e-33f5-49e2-8b93-1a97dc9dbf04`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:52,502::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:40:52,502::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422533', 'lastCheck': '1.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:52,502::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9d9ed0e-33f5-49e2-8b93-1a97dc9dbf04`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422533', 'lastCheck': '1.1', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:40:52,502::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9d9ed0e-33f5-49e2-8b93-1a97dc9dbf04`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:52,503::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:52,503::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:52,503::task::993::Storage.TaskManager.Task::(_decref) Task=`b9d9ed0e-33f5-49e2-8b93-1a97dc9dbf04`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:40:52,521::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:40:54,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:54,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013162 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:40:56,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:56,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132557 s, 77.2 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:40:57,423::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:40:57,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5b8f5a4-1b7d-41c8-a98c-5c40cf8e7c83`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:40:57,454::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:57,462::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:57,464::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:40:57,465::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:40:57,468::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:57,468::task::1191::Storage.TaskManager.Task::(prepare) Task=`a5b8f5a4-1b7d-41c8-a98c-5c40cf8e7c83`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:40:57,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5b8f5a4-1b7d-41c8-a98c-5c40cf8e7c83`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:40:57,469::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:40:57,469::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:40:57,469::task::993::Storage.TaskManager.Task::(_decref) Task=`a5b8f5a4-1b7d-41c8-a98c-5c40cf8e7c83`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:40:57,479::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:40:57,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`7da04427-fc9e-472f-a930-28a39a44dcc6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:40:57,481::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:40:57,481::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bd548646-9c65-48f5-81d1-7299490fb65f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:40:57,482::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:40:57,482::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:40:57,482::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bd548646-9c65-48f5-81d1-7299490fb65f`::Granted request >Thread-34::DEBUG::2014-08-11 14:40:57,483::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7da04427-fc9e-472f-a930-28a39a44dcc6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:40:57,483::task::993::Storage.TaskManager.Task::(_decref) Task=`7da04427-fc9e-472f-a930-28a39a44dcc6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:40:57,503::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:57,503::task::1191::Storage.TaskManager.Task::(prepare) Task=`7da04427-fc9e-472f-a930-28a39a44dcc6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:40:57,503::task::595::Storage.TaskManager.Task::(_updateState) Task=`7da04427-fc9e-472f-a930-28a39a44dcc6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:40:57,503::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:40:57,504::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:40:57,504::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:40:57,504::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:40:57,505::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:40:57,505::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:40:57,505::task::993::Storage.TaskManager.Task::(_decref) Task=`7da04427-fc9e-472f-a930-28a39a44dcc6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:40:58,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:40:58,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132893 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:00,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:00,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133566 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:01,440::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:01,441::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:41:01,442::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:41:01,442::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:01,460::__init__::232::IOProcess::(_processLogs) (5502) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:01,461::__init__::232::IOProcess::(_processLogs) (5502) Queuing response >Thread-16::DEBUG::2014-08-11 14:41:01,462::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000568536 s, 800 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:01,463::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:01,463::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:01,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:01,463::__init__::232::IOProcess::(_processLogs) (5503) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:01,464::__init__::232::IOProcess::(_processLogs) DEBUG|(5503) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:01,466::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:01,466::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:01,467::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:01,467::__init__::232::IOProcess::(_processLogs) (5504) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:01,468::__init__::232::IOProcess::(_processLogs) (5504) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:01,468::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:01,469::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:01,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:01,470::__init__::232::IOProcess::(_processLogs) (5505) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:01,470::__init__::232::IOProcess::(_processLogs) (5505) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:01,470::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:01,471::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:01,471::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:01,472::__init__::232::IOProcess::(_processLogs) (5506) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:01,473::__init__::232::IOProcess::(_processLogs) (5506) Queuing response >Dummy-33::DEBUG::2014-08-11 14:41:02,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:02,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223815 s, 45.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:02,473::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:02,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:02,558::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:04,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:04,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132549 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:06,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:06,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132655 s, 77.2 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:41:07,516::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:41:07,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f6fd793-07f8-47dc-95bf-96479eec2a36`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:41:07,554::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:41:07,567::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:41:07,570::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f6fd793-07f8-47dc-95bf-96479eec2a36`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:07,571::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:41:07,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f6fd793-07f8-47dc-95bf-96479eec2a36`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:41:07,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:41:07,573::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:41:07,575::task::993::Storage.TaskManager.Task::(_decref) Task=`3f6fd793-07f8-47dc-95bf-96479eec2a36`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:07,576::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:07,580::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:41:07,589::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:41:07,590::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a1d78b6-3af9-435c-a9f7-57c5b4fe2b7e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:41:07,590::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:41:07,591::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`01b09d64-a3d7-4dc2-a2e2-ea35fac01c5f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:41:07,591::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:41:07,592::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:41:07,592::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`01b09d64-a3d7-4dc2-a2e2-ea35fac01c5f`::Granted request >Thread-34::DEBUG::2014-08-11 14:41:07,592::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2a1d78b6-3af9-435c-a9f7-57c5b4fe2b7e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:41:07,593::task::993::Storage.TaskManager.Task::(_decref) Task=`2a1d78b6-3af9-435c-a9f7-57c5b4fe2b7e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:41:07,612::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:07,613::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a1d78b6-3af9-435c-a9f7-57c5b4fe2b7e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:07,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a1d78b6-3af9-435c-a9f7-57c5b4fe2b7e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:41:07,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:41:07,613::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:41:07,614::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:41:07,614::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:41:07,614::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:41:07,615::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:41:07,615::task::993::Storage.TaskManager.Task::(_decref) Task=`2a1d78b6-3af9-435c-a9f7-57c5b4fe2b7e`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:41:07,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ed0523e-f6bd-4855-b824-b039ff3ab7b1`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:41:07,622::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:41:07,623::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000568536', 'lastCheck': '6.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:41:07,623::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ed0523e-f6bd-4855-b824-b039ff3ab7b1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000568536', 'lastCheck': '6.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:41:07,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ed0523e-f6bd-4855-b824-b039ff3ab7b1`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:41:07,623::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:41:07,624::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:41:07,624::task::993::Storage.TaskManager.Task::(_decref) Task=`9ed0523e-f6bd-4855-b824-b039ff3ab7b1`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:41:07,646::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:08,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:08,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132346 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:10,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:10,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132716 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:11,473::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:11,474::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:11,474::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:11,474::__init__::232::IOProcess::(_processLogs) (5507) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:11,475::__init__::232::IOProcess::(_processLogs) DEBUG|(5507) Queuing response >Thread-16::DEBUG::2014-08-11 14:41:11,476::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:41:11,495::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000410214 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:11,496::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:11,497::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:11,497::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:11,497::__init__::232::IOProcess::(_processLogs) (5508) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:11,498::__init__::232::IOProcess::(_processLogs) DEBUG|(5508) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:11,499::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:11,500::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:11,500::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:11,500::__init__::232::IOProcess::(_processLogs) (5509) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:11,501::__init__::232::IOProcess::(_processLogs) DEBUG|(5509) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:11,502::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:11,502::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:11,503::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:11,503::__init__::232::IOProcess::(_processLogs) (5510) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:11,503::__init__::232::IOProcess::(_processLogs) (5510) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:11,504::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:11,504::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:11,504::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:11,504::__init__::232::IOProcess::(_processLogs) (5511) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:11,505::__init__::232::IOProcess::(_processLogs) DEBUG|(5511) Queuing response >Thread-55::DEBUG::2014-08-11 14:41:12,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`24ee8658-c7e7-4261-99cb-f96959221c63`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:41:12,210::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:41:12,210::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:12,211::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:12,211::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:12,211::__init__::232::IOProcess::(_processLogs) (5512) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:41:12,212::__init__::232::IOProcess::(_processLogs) DEBUG|(5512) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:12,213::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:12,213::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:41:12,214::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:41:12,215::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:41:12,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`24ee8658-c7e7-4261-99cb-f96959221c63`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:41:12,216::__init__::232::IOProcess::(_processLogs) (5513) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:41:12,217::task::595::Storage.TaskManager.Task::(_updateState) Task=`24ee8658-c7e7-4261-99cb-f96959221c63`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:41:12,217::__init__::232::IOProcess::(_processLogs) (5513) Queuing response >Thread-55::DEBUG::2014-08-11 14:41:12,218::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:41:12,219::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:41:12,219::task::993::Storage.TaskManager.Task::(_decref) Task=`24ee8658-c7e7-4261-99cb-f96959221c63`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:12,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:12,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:12,586::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:12,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:12,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134844 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:14,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:14,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131158 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:16,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:16,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013154 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:17,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:17,592::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:41:17,629::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:41:17,657::task::595::Storage.TaskManager.Task::(_updateState) Task=`4eb4fe7f-11ad-4f2d-9eb7-b05d7aa8934a`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:41:17,659::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:17,659::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::INFO::2014-08-11 14:41:17,673::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:41:17,673::task::1191::Storage.TaskManager.Task::(prepare) Task=`4eb4fe7f-11ad-4f2d-9eb7-b05d7aa8934a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:41:17,673::task::595::Storage.TaskManager.Task::(_updateState) Task=`4eb4fe7f-11ad-4f2d-9eb7-b05d7aa8934a`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:41:17,674::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:41:17,674::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:41:17,674::task::993::Storage.TaskManager.Task::(_decref) Task=`4eb4fe7f-11ad-4f2d-9eb7-b05d7aa8934a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:41:17,684::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:41:17,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`960a3593-1835-4413-8f02-4317ca5e2706`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:41:17,685::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:41:17,686::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d58b16fa-55b3-45db-a1c6-6f6c37a1b004`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:41:17,686::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:41:17,686::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:41:17,686::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d58b16fa-55b3-45db-a1c6-6f6c37a1b004`::Granted request >Thread-34::DEBUG::2014-08-11 14:41:17,689::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`960a3593-1835-4413-8f02-4317ca5e2706`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:41:17,689::task::993::Storage.TaskManager.Task::(_decref) Task=`960a3593-1835-4413-8f02-4317ca5e2706`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:41:17,709::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:17,710::task::1191::Storage.TaskManager.Task::(prepare) Task=`960a3593-1835-4413-8f02-4317ca5e2706`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861129601024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:17,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`960a3593-1835-4413-8f02-4317ca5e2706`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:41:17,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:41:17,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:41:17,711::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:41:17,711::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:41:17,711::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:41:17,711::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:41:17,712::task::993::Storage.TaskManager.Task::(_decref) Task=`960a3593-1835-4413-8f02-4317ca5e2706`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:41:18,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:18,960::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131759 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:20,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:21,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133376 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:21,506::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:21,507::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:21,508::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:41:21,509::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:41:21,509::__init__::232::IOProcess::(_processLogs) (5514) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:21,510::__init__::232::IOProcess::(_processLogs) (5514) Queuing response >Thread-16::DEBUG::2014-08-11 14:41:21,529::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000476971 s, 954 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:21,530::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:21,530::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:21,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:21,531::__init__::232::IOProcess::(_processLogs) (5515) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:21,531::__init__::232::IOProcess::(_processLogs) DEBUG|(5515) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:21,533::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:21,533::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:21,534::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:21,534::__init__::232::IOProcess::(_processLogs) (5516) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:21,534::__init__::232::IOProcess::(_processLogs) (5516) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:21,535::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:21,535::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:21,535::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:21,536::__init__::232::IOProcess::(_processLogs) (5517) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:21,536::__init__::232::IOProcess::(_processLogs) DEBUG|(5517) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:21,537::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:21,538::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:21,538::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:21,539::__init__::232::IOProcess::(_processLogs) (5518) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:21,539::__init__::232::IOProcess::(_processLogs) (5518) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:22,667::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:22,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:41:22,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`d19ffcae-966c-4f04-8e0a-1bf8fcafebb2`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:41:22,755::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-95::INFO::2014-08-11 14:41:22,757::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000476971', 'lastCheck': '1.2', 'valid': True}} >Thread-95::DEBUG::2014-08-11 14:41:22,759::task::1191::Storage.TaskManager.Task::(prepare) Task=`d19ffcae-966c-4f04-8e0a-1bf8fcafebb2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000476971', 'lastCheck': '1.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:22,759::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:41:22,760::task::595::Storage.TaskManager.Task::(_updateState) Task=`d19ffcae-966c-4f04-8e0a-1bf8fcafebb2`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:41:22,761::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:41:22,762::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:41:22,764::task::993::Storage.TaskManager.Task::(_decref) Task=`d19ffcae-966c-4f04-8e0a-1bf8fcafebb2`::ref 0 aborting False >Thread-95::DEBUG::2014-08-11 14:41:22,784::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:23,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:23,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131303 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:25,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:25,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132143 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:27,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:27,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131863 s, 77.7 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:41:27,726::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:41:27,727::task::595::Storage.TaskManager.Task::(_updateState) Task=`02ee596d-08ec-4088-8b57-99a3691dc2dd`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:41:27,754::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:41:27,767::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:41:27,768::task::1191::Storage.TaskManager.Task::(prepare) Task=`02ee596d-08ec-4088-8b57-99a3691dc2dd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:41:27,768::task::595::Storage.TaskManager.Task::(_updateState) Task=`02ee596d-08ec-4088-8b57-99a3691dc2dd`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:41:27,768::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:41:27,768::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:41:27,768::task::993::Storage.TaskManager.Task::(_decref) Task=`02ee596d-08ec-4088-8b57-99a3691dc2dd`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:27,771::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:27,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:27,774::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:41:27,785::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:41:27,785::task::595::Storage.TaskManager.Task::(_updateState) Task=`274a1c3c-2826-4547-a7de-a10f594fbdd2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:41:27,786::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:41:27,786::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7ed02f4-5dd3-4ca7-a9da-7be7fe7132c3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:41:27,786::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:41:27,787::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:41:27,787::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7ed02f4-5dd3-4ca7-a9da-7be7fe7132c3`::Granted request >Thread-34::DEBUG::2014-08-11 14:41:27,788::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`274a1c3c-2826-4547-a7de-a10f594fbdd2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:41:27,788::task::993::Storage.TaskManager.Task::(_decref) Task=`274a1c3c-2826-4547-a7de-a10f594fbdd2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:41:27,808::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861127897088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:27,808::task::1191::Storage.TaskManager.Task::(prepare) Task=`274a1c3c-2826-4547-a7de-a10f594fbdd2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861127897088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:27,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`274a1c3c-2826-4547-a7de-a10f594fbdd2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:41:27,809::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:41:27,809::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:41:27,809::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:41:27,810::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:41:27,810::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:41:27,810::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:41:27,810::task::993::Storage.TaskManager.Task::(_decref) Task=`274a1c3c-2826-4547-a7de-a10f594fbdd2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:41:29,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:29,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133315 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:31,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:31,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132831 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:31,540::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:31,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:31,541::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:41:31,542::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:41:31,543::__init__::232::IOProcess::(_processLogs) (5519) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:31,560::__init__::232::IOProcess::(_processLogs) (5519) Queuing response >Thread-16::DEBUG::2014-08-11 14:41:31,562::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000482198 s, 944 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:31,563::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:31,564::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:31,564::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:31,564::__init__::232::IOProcess::(_processLogs) (5520) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:31,564::__init__::232::IOProcess::(_processLogs) (5520) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:31,565::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:31,566::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:31,567::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:31,567::__init__::232::IOProcess::(_processLogs) (5521) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:31,567::__init__::232::IOProcess::(_processLogs) (5521) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:31,568::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:31,569::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:31,569::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:31,569::__init__::232::IOProcess::(_processLogs) (5522) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:31,570::__init__::232::IOProcess::(_processLogs) (5522) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:31,570::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:31,571::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:31,571::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:31,572::__init__::232::IOProcess::(_processLogs) (5523) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:31,572::__init__::232::IOProcess::(_processLogs) (5523) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:32,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:32,788::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:32,859::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:33,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:33,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013239 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:35,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:35,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207188 s, 49.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:37,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:37,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132524 s, 77.3 MB/s\n'; <rc> = 0 >Thread-95::DEBUG::2014-08-11 14:41:37,822::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-95::DEBUG::2014-08-11 14:41:37,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ec880b1-9770-4adf-b9f8-b6a1a195c7c9`::moving from state init -> state preparing >Thread-95::INFO::2014-08-11 14:41:37,854::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-95::INFO::2014-08-11 14:41:37,868::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:37,870::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-95::DEBUG::2014-08-11 14:41:37,870::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ec880b1-9770-4adf-b9f8-b6a1a195c7c9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-95::DEBUG::2014-08-11 14:41:37,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ec880b1-9770-4adf-b9f8-b6a1a195c7c9`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-08-11 14:41:37,873::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-08-11 14:41:37,874::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-08-11 14:41:37,875::task::993::Storage.TaskManager.Task::(_decref) Task=`7ec880b1-9770-4adf-b9f8-b6a1a195c7c9`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:37,876::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:37,879::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:41:37,899::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:41:37,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`2507536a-1ee5-49ad-ac7d-1cae52222726`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:41:37,900::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:41:37,901::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1900a6fa-36b5-4a3b-b879-c2654fc5d5f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:41:37,901::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:41:37,901::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:41:37,902::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1900a6fa-36b5-4a3b-b879-c2654fc5d5f9`::Granted request >Thread-34::DEBUG::2014-08-11 14:41:37,902::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2507536a-1ee5-49ad-ac7d-1cae52222726`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:41:37,902::task::993::Storage.TaskManager.Task::(_decref) Task=`2507536a-1ee5-49ad-ac7d-1cae52222726`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:41:37,922::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861106794496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:37,922::task::1191::Storage.TaskManager.Task::(prepare) Task=`2507536a-1ee5-49ad-ac7d-1cae52222726`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861106794496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:37,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`2507536a-1ee5-49ad-ac7d-1cae52222726`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:41:37,923::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:41:37,923::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:41:37,923::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:41:37,924::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:41:37,924::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:41:37,924::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:41:37,925::task::993::Storage.TaskManager.Task::(_decref) Task=`2507536a-1ee5-49ad-ac7d-1cae52222726`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 14:41:38,094::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:53687 >Detector thread::DEBUG::2014-08-11 14:41:38,099::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:53687 >Detector thread::DEBUG::2014-08-11 14:41:38,099::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:53687 >Detector thread::DEBUG::2014-08-11 14:41:38,100::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 53687) >Thread-97::DEBUG::2014-08-11 14:41:38,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`103e8667-168d-49fd-bf57-a306f0e97736`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:41:38,103::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:41:38,103::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000482198', 'lastCheck': '6.5', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:41:38,104::task::1191::Storage.TaskManager.Task::(prepare) Task=`103e8667-168d-49fd-bf57-a306f0e97736`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000482198', 'lastCheck': '6.5', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:41:38,104::task::595::Storage.TaskManager.Task::(_updateState) Task=`103e8667-168d-49fd-bf57-a306f0e97736`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:41:38,104::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:41:38,104::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:41:38,104::task::993::Storage.TaskManager.Task::(_decref) Task=`103e8667-168d-49fd-bf57-a306f0e97736`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:41:38,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:39,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:39,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171491 s, 59.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:41,573::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:41,574::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 14:41:41,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:41:41,576::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:41:41,593::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:41:41,595::__init__::232::IOProcess::(_processLogs) (5524) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:41,616::__init__::232::IOProcess::(_processLogs) (5524) Queuing response >Thread-16::DEBUG::2014-08-11 14:41:41,618::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000505133 s, 901 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:41,624::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:41,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:41,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:41,624::__init__::232::IOProcess::(_processLogs) (5525) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:41,626::__init__::232::IOProcess::(_processLogs) DEBUG|(5525) Queuing response >Dummy-33::DEBUG::2014-08-11 14:41:41,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336349 s, 30.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:41,633::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:41,635::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:41,636::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:41,636::__init__::232::IOProcess::(_processLogs) (5526) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:41,637::__init__::232::IOProcess::(_processLogs) (5526) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:41,640::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:41,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:41,642::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:41,642::__init__::232::IOProcess::(_processLogs) (5527) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:41,643::__init__::232::IOProcess::(_processLogs) (5527) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:41,645::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:41,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:41,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:41,649::__init__::232::IOProcess::(_processLogs) (5528) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:41,650::__init__::232::IOProcess::(_processLogs) (5528) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:42,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:42,887::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:42,957::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:43,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:43,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013284 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:45,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:45,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131681 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:47,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:47,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133264 s, 76.8 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:41:47,936::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:41:47,937::task::595::Storage.TaskManager.Task::(_updateState) Task=`88e0a593-92ca-4414-9a7a-96a567052cae`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:41:47,956::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:47,965::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:47,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:47,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:41:47,971::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:41:47,971::task::1191::Storage.TaskManager.Task::(prepare) Task=`88e0a593-92ca-4414-9a7a-96a567052cae`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:41:47,971::task::595::Storage.TaskManager.Task::(_updateState) Task=`88e0a593-92ca-4414-9a7a-96a567052cae`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:41:47,971::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:41:47,972::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:41:47,972::task::993::Storage.TaskManager.Task::(_decref) Task=`88e0a593-92ca-4414-9a7a-96a567052cae`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:41:47,986::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:41:47,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`e198d438-de77-4dec-b822-45fa0368bd74`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:41:47,987::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:41:47,987::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c9907053-928f-42a9-afd2-813d1dfb69b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:41:47,988::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:41:47,988::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:41:47,988::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c9907053-928f-42a9-afd2-813d1dfb69b6`::Granted request >Thread-34::DEBUG::2014-08-11 14:41:47,989::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e198d438-de77-4dec-b822-45fa0368bd74`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:41:47,989::task::993::Storage.TaskManager.Task::(_decref) Task=`e198d438-de77-4dec-b822-45fa0368bd74`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:41:48,009::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861106663424', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:48,009::task::1191::Storage.TaskManager.Task::(prepare) Task=`e198d438-de77-4dec-b822-45fa0368bd74`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861106663424', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:48,009::task::595::Storage.TaskManager.Task::(_updateState) Task=`e198d438-de77-4dec-b822-45fa0368bd74`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:41:48,010::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:41:48,010::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:41:48,010::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:41:48,011::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:41:48,011::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:41:48,011::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:41:48,011::task::993::Storage.TaskManager.Task::(_decref) Task=`e198d438-de77-4dec-b822-45fa0368bd74`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:41:49,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:49,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134227 s, 76.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:51,647::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:51,648::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:51,648::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:51,648::__init__::232::IOProcess::(_processLogs) (5529) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:51,649::__init__::232::IOProcess::(_processLogs) DEBUG|(5529) Queuing response >Thread-16::DEBUG::2014-08-11 14:41:51,650::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:41:51,668::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000435688 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:41:51,669::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:51,670::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:51,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:51,670::__init__::232::IOProcess::(_processLogs) (5530) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:41:51,671::__init__::232::IOProcess::(_processLogs) DEBUG|(5530) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:51,672::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:51,673::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:51,674::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:51,674::__init__::232::IOProcess::(_processLogs) (5531) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:51,674::__init__::232::IOProcess::(_processLogs) (5531) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:51,674::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:51,675::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:51,676::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:51,676::__init__::232::IOProcess::(_processLogs) (5532) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:51,676::__init__::232::IOProcess::(_processLogs) (5532) Queuing response >Thread-14::DEBUG::2014-08-11 14:41:51,677::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:41:51,677::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:41:51,678::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:41:51,679::__init__::232::IOProcess::(_processLogs) (5533) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:41:51,679::__init__::232::IOProcess::(_processLogs) (5533) Queuing response >Dummy-33::DEBUG::2014-08-11 14:41:51,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:51,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132936 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:52,976::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:52,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:53,057::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:41:53,238::task::595::Storage.TaskManager.Task::(_updateState) Task=`b73f4613-e29c-4b52-9a4d-31cdf28138df`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:41:53,238::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:41:53,238::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435688', 'lastCheck': '1.6', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:41:53,239::task::1191::Storage.TaskManager.Task::(prepare) Task=`b73f4613-e29c-4b52-9a4d-31cdf28138df`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435688', 'lastCheck': '1.6', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:41:53,239::task::595::Storage.TaskManager.Task::(_updateState) Task=`b73f4613-e29c-4b52-9a4d-31cdf28138df`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:41:53,239::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:41:53,239::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:41:53,240::task::993::Storage.TaskManager.Task::(_decref) Task=`b73f4613-e29c-4b52-9a4d-31cdf28138df`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:41:53,259::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:53,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:53,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132379 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:41:55,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:41:56,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133687 s, 76.6 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:41:58,024::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:41:58,025::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c918d6e-6281-4a0a-a8e6-21b89d17683b`::moving from state init -> state preparing >Dummy-33::DEBUG::2014-08-11 14:41:58,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-97::INFO::2014-08-11 14:41:58,057::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:58,077::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:58,081::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:41:58,083::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:41:58,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013278 s, 77.1 MB/s\n'; <rc> = 0 >Thread-97::INFO::2014-08-11 14:41:58,103::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:41:58,105::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c918d6e-6281-4a0a-a8e6-21b89d17683b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:41:58,106::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c918d6e-6281-4a0a-a8e6-21b89d17683b`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:41:58,107::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:41:58,107::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:41:58,109::task::993::Storage.TaskManager.Task::(_decref) Task=`6c918d6e-6281-4a0a-a8e6-21b89d17683b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:41:58,121::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:41:58,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`752f696a-5b7b-4be2-9873-6e4119f4e45e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:41:58,122::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:41:58,122::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7b15a0dd-40cb-4976-9df8-fb711301b5a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:41:58,123::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:41:58,123::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:41:58,123::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7b15a0dd-40cb-4976-9df8-fb711301b5a1`::Granted request >Thread-34::DEBUG::2014-08-11 14:41:58,124::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`752f696a-5b7b-4be2-9873-6e4119f4e45e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:41:58,124::task::993::Storage.TaskManager.Task::(_decref) Task=`752f696a-5b7b-4be2-9873-6e4119f4e45e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:41:58,144::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861091721216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:58,144::task::1191::Storage.TaskManager.Task::(prepare) Task=`752f696a-5b7b-4be2-9873-6e4119f4e45e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861091721216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:41:58,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`752f696a-5b7b-4be2-9873-6e4119f4e45e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:41:58,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:41:58,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:41:58,145::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:41:58,146::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:41:58,146::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:41:58,146::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:41:58,147::task::993::Storage.TaskManager.Task::(_decref) Task=`752f696a-5b7b-4be2-9873-6e4119f4e45e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:42:00,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:00,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134224 s, 76.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:01,679::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:01,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:01,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:42:01,682::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:42:01,683::__init__::232::IOProcess::(_processLogs) (5534) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:01,701::__init__::232::IOProcess::(_processLogs) (5534) Queuing response >Thread-16::DEBUG::2014-08-11 14:42:01,702::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00056883 s, 800 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:01,703::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:01,703::__init__::232::IOProcess::(_processLogs) (5535) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:01,705::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:01,705::__init__::232::IOProcess::(_processLogs) (5536) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:01,706::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:01,706::__init__::232::IOProcess::(_processLogs) (5537) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:01,706::__init__::232::IOProcess::(_processLogs) (5537) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:01,707::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:01,707::__init__::232::IOProcess::(_processLogs) (5538) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:01,708::__init__::232::IOProcess::(_processLogs) (5538) Queuing response >Dummy-33::DEBUG::2014-08-11 14:42:02,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:02,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299933 s, 34.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:03,090::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:03,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:03,157::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:42:04,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:04,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132675 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:06,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:06,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135106 s, 75.8 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:42:08,159::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:42:08,160::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a6af299-d2ac-45c4-ba8d-4e60fbf69849`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:08,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:42:08,257::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:08,262::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:08,264::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:42:08,275::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:08,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a6af299-d2ac-45c4-ba8d-4e60fbf69849`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:08,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a6af299-d2ac-45c4-ba8d-4e60fbf69849`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:08,279::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:08,280::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:08,281::task::993::Storage.TaskManager.Task::(_decref) Task=`8a6af299-d2ac-45c4-ba8d-4e60fbf69849`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:42:08,298::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:42:08,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d033e1d-5b17-417d-bbba-fa7d5671ba72`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:42:08,303::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:42:08,306::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`22f8e639-7715-4b19-a0e3-d1de32d1fa07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:42:08,307::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:42:08,309::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:42:08,310::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`22f8e639-7715-4b19-a0e3-d1de32d1fa07`::Granted request >Thread-34::DEBUG::2014-08-11 14:42:08,314::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6d033e1d-5b17-417d-bbba-fa7d5671ba72`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:42:08,316::task::993::Storage.TaskManager.Task::(_decref) Task=`6d033e1d-5b17-417d-bbba-fa7d5671ba72`::ref 1 aborting False >Thread-97::DEBUG::2014-08-11 14:42:08,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`2eb845cf-7541-4507-b5ce-9ab432c9fa91`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:42:08,357::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:42:08,363::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00056883', 'lastCheck': '6.7', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:42:08,367::task::1191::Storage.TaskManager.Task::(prepare) Task=`2eb845cf-7541-4507-b5ce-9ab432c9fa91`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00056883', 'lastCheck': '6.7', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:42:08,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`2eb845cf-7541-4507-b5ce-9ab432c9fa91`::moving from state preparing -> state finished >Thread-34::INFO::2014-08-11 14:42:08,370::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861061771264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-97::DEBUG::2014-08-11 14:42:08,370::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-34::DEBUG::2014-08-11 14:42:08,371::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d033e1d-5b17-417d-bbba-fa7d5671ba72`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861061771264', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-97::DEBUG::2014-08-11 14:42:08,372::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:42:08,373::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d033e1d-5b17-417d-bbba-fa7d5671ba72`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:08,373::task::993::Storage.TaskManager.Task::(_decref) Task=`2eb845cf-7541-4507-b5ce-9ab432c9fa91`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:42:08,374::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:42:08,376::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:42:08,380::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:42:08,382::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:42:08,383::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:42:08,385::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:42:08,387::task::993::Storage.TaskManager.Task::(_decref) Task=`6d033e1d-5b17-417d-bbba-fa7d5671ba72`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:42:08,417::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:42:08,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:08,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132584 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:10,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:10,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131348 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:11,710::__init__::232::IOProcess::(_processLogs) (5539) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:42:11,711::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:42:11,729::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412588 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:11,731::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:11,731::__init__::232::IOProcess::(_processLogs) (5540) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:11,732::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:11,732::__init__::232::IOProcess::(_processLogs) (5541) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:11,733::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:11,734::__init__::232::IOProcess::(_processLogs) (5542) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:11,734::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:11,735::__init__::232::IOProcess::(_processLogs) (5543) Got request for method 'access' >Thread-55::DEBUG::2014-08-11 14:42:12,323::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcf53f53-8050-45ce-82d8-47fe4af3a8fc`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:42:12,324::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:42:12,324::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:12,325::__init__::232::IOProcess::(_processLogs) (5544) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:42:12,325::__init__::232::IOProcess::(_processLogs) DEBUG|(5544) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:12,326::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:12,327::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:42:12,327::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:42:12,328::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:42:12,329::task::1191::Storage.TaskManager.Task::(prepare) Task=`dcf53f53-8050-45ce-82d8-47fe4af3a8fc`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:42:12,329::__init__::232::IOProcess::(_processLogs) (5545) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:42:12,330::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcf53f53-8050-45ce-82d8-47fe4af3a8fc`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:42:12,331::__init__::232::IOProcess::(_processLogs) (5545) Queuing response >Thread-55::DEBUG::2014-08-11 14:42:12,331::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:42:12,332::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:42:12,332::task::993::Storage.TaskManager.Task::(_decref) Task=`dcf53f53-8050-45ce-82d8-47fe4af3a8fc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:42:12,709::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:12,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013272 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:13,273::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:13,274::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:13,357::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:42:14,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:14,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132065 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:16,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:16,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013225 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:18,360::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:18,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:18,404::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:42:18,459::task::595::Storage.TaskManager.Task::(_updateState) Task=`52d7c9f2-3f58-4483-b60b-7083e79bff5f`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:42:18,461::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:18,463::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:42:18,480::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:18,482::task::1191::Storage.TaskManager.Task::(prepare) Task=`52d7c9f2-3f58-4483-b60b-7083e79bff5f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:18,483::task::595::Storage.TaskManager.Task::(_updateState) Task=`52d7c9f2-3f58-4483-b60b-7083e79bff5f`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:18,484::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:18,485::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:18,487::task::993::Storage.TaskManager.Task::(_decref) Task=`52d7c9f2-3f58-4483-b60b-7083e79bff5f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:42:18,502::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:42:18,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d7e3e4f-61cb-4571-bcc7-00f7210264d7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:42:18,506::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:42:18,510::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7b79a4f-b512-43a9-a5f1-dfc1c892f4b5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:42:18,511::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:42:18,513::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:42:18,514::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7b79a4f-b512-43a9-a5f1-dfc1c892f4b5`::Granted request >Thread-34::DEBUG::2014-08-11 14:42:18,518::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5d7e3e4f-61cb-4571-bcc7-00f7210264d7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:42:18,520::task::993::Storage.TaskManager.Task::(_decref) Task=`5d7e3e4f-61cb-4571-bcc7-00f7210264d7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:42:18,563::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861048074240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:18,564::task::1191::Storage.TaskManager.Task::(prepare) Task=`5d7e3e4f-61cb-4571-bcc7-00f7210264d7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861048074240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:18,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d7e3e4f-61cb-4571-bcc7-00f7210264d7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:42:18,564::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:42:18,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:42:18,565::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:42:18,565::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:42:18,566::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:42:18,566::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:42:18,566::task::993::Storage.TaskManager.Task::(_decref) Task=`5d7e3e4f-61cb-4571-bcc7-00f7210264d7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:42:18,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:18,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132429 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:20,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:20,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:21,737::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:21,737::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:21,737::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:21,738::__init__::232::IOProcess::(_processLogs) (5546) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:21,739::__init__::232::IOProcess::(_processLogs) DEBUG|(5546) Queuing response >Thread-16::DEBUG::2014-08-11 14:42:21,740::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:42:21,758::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426128 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:21,759::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:21,760::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:21,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:21,760::__init__::232::IOProcess::(_processLogs) (5547) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:21,761::__init__::232::IOProcess::(_processLogs) DEBUG|(5547) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:21,762::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:21,763::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:21,764::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:21,764::__init__::232::IOProcess::(_processLogs) (5548) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:21,764::__init__::232::IOProcess::(_processLogs) (5548) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:21,764::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:21,765::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:21,765::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:21,765::__init__::232::IOProcess::(_processLogs) (5549) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:21,766::__init__::232::IOProcess::(_processLogs) DEBUG|(5549) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:21,767::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:21,767::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:21,768::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:21,769::__init__::232::IOProcess::(_processLogs) (5550) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:21,769::__init__::232::IOProcess::(_processLogs) (5550) Queuing response >Dummy-33::DEBUG::2014-08-11 14:42:22,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:23,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131846 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:23,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:23,473::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:23,509::task::595::Storage.TaskManager.Task::(_updateState) Task=`08bd2193-14d9-489b-b99a-aca6cdd2a59d`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:42:23,558::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:42:23,559::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426128', 'lastCheck': '1.8', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:23,561::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:23,561::task::1191::Storage.TaskManager.Task::(prepare) Task=`08bd2193-14d9-489b-b99a-aca6cdd2a59d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426128', 'lastCheck': '1.8', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:42:23,563::task::595::Storage.TaskManager.Task::(_updateState) Task=`08bd2193-14d9-489b-b99a-aca6cdd2a59d`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:23,564::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:23,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:23,565::task::993::Storage.TaskManager.Task::(_decref) Task=`08bd2193-14d9-489b-b99a-aca6cdd2a59d`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:42:23,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:42:25,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:25,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132877 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:27,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:27,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131663 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:28,568::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:28,569::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:28,570::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:28,580::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:42:28,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb8e18fe-f401-450e-83ce-63fdcd085d3d`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:42:28,584::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:42:28,598::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:28,600::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb8e18fe-f401-450e-83ce-63fdcd085d3d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:28,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb8e18fe-f401-450e-83ce-63fdcd085d3d`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:28,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:28,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:28,606::task::993::Storage.TaskManager.Task::(_decref) Task=`bb8e18fe-f401-450e-83ce-63fdcd085d3d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:42:28,623::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:42:28,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`1141160f-24bc-4d53-bf40-9fc075ae387e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:42:28,627::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:42:28,631::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2364e39f-22c2-4dff-8bc4-4b6fed435e20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:42:28,632::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:42:28,634::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:42:28,636::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2364e39f-22c2-4dff-8bc4-4b6fed435e20`::Granted request >Thread-34::DEBUG::2014-08-11 14:42:28,638::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1141160f-24bc-4d53-bf40-9fc075ae387e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:42:28,640::task::993::Storage.TaskManager.Task::(_decref) Task=`1141160f-24bc-4d53-bf40-9fc075ae387e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:42:28,692::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861045846016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:28,693::task::1191::Storage.TaskManager.Task::(prepare) Task=`1141160f-24bc-4d53-bf40-9fc075ae387e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861045846016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:28,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`1141160f-24bc-4d53-bf40-9fc075ae387e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:42:28,695::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:42:28,696::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:42:28,698::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:42:28,700::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:42:28,701::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:42:28,702::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:42:28,704::task::993::Storage.TaskManager.Task::(_decref) Task=`1141160f-24bc-4d53-bf40-9fc075ae387e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:42:29,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:29,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132616 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:31,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:31,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137554 s, 74.4 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:42:31,769::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:42:31,771::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,771::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,772::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,774::__init__::232::IOProcess::(_processLogs) (5551) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:42:31,774::__init__::232::IOProcess::(_processLogs) (5551) Queuing response >Thread-16::DEBUG::2014-08-11 14:42:31,781::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:42:31,782::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,782::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,783::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,783::__init__::232::IOProcess::(_processLogs) (5552) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 14:42:31,783::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5552) Queuing response >Thread-16::DEBUG::2014-08-11 14:42:31,784::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:42:31,785::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,785::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,785::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,786::__init__::232::IOProcess::(_processLogs) (5553) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:42:31,786::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5553) Queuing response >Thread-16::DEBUG::2014-08-11 14:42:31,786::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:42:31,788::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,789::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,790::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,790::__init__::232::IOProcess::(_processLogs) (5554) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:31,790::__init__::232::IOProcess::(_processLogs) (5554) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:31,791::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,791::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:42:31,791::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 14:42:31,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::INFO::2014-08-11 14:42:31,793::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-14::DEBUG::2014-08-11 14:42:31,794::__init__::232::IOProcess::(_processLogs) (5555) Got request for method 'glob' >Thread-16::INFO::2014-08-11 14:42:31,794::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:42:31,795::__init__::232::IOProcess::(_processLogs) (5555) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:31,796::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,797::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,797::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,797::__init__::232::IOProcess::(_processLogs) (5556) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:31,798::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5556) Queuing response >Thread-16::DEBUG::2014-08-11 14:42:31,799::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:42:31,817::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418426 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:31,818::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,818::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,819::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,819::__init__::232::IOProcess::(_processLogs) (5557) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:31,820::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5557) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:31,821::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,821::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,822::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,822::__init__::232::IOProcess::(_processLogs) (5558) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:31,822::__init__::232::IOProcess::(_processLogs) (5558) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:31,823::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,823::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,824::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,824::__init__::232::IOProcess::(_processLogs) (5559) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:31,825::__init__::232::IOProcess::(_processLogs) (5559) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:31,825::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:31,825::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:31,826::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:31,826::__init__::232::IOProcess::(_processLogs) (5560) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:31,826::__init__::232::IOProcess::(_processLogs) (5560) Queuing response >Dummy-33::DEBUG::2014-08-11 14:42:33,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:33,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132485 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:33,575::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:33,576::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:33,657::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:42:35,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:35,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135091 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:37,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:37,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132653 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:38,664::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:38,665::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:38,685::task::595::Storage.TaskManager.Task::(_updateState) Task=`469d8e52-c81e-4c7a-9a31-5967e7ff6472`::moving from state init -> state preparing >Detector thread::DEBUG::2014-08-11 14:42:38,721::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:54555 >Thread-97::INFO::2014-08-11 14:42:38,759::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Detector thread::DEBUG::2014-08-11 14:42:38,761::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:54555 >Thread-97::INFO::2014-08-11 14:42:38,764::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418426', 'lastCheck': '6.9', 'valid': True}} >Detector thread::DEBUG::2014-08-11 14:42:38,764::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:54555 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:38,766::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:38,767::task::1191::Storage.TaskManager.Task::(prepare) Task=`469d8e52-c81e-4c7a-9a31-5967e7ff6472`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418426', 'lastCheck': '6.9', 'valid': True}} >Detector thread::DEBUG::2014-08-11 14:42:38,768::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 54555) >Thread-97::DEBUG::2014-08-11 14:42:38,771::task::595::Storage.TaskManager.Task::(_updateState) Task=`469d8e52-c81e-4c7a-9a31-5967e7ff6472`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:38,774::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:38,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:38,778::task::993::Storage.TaskManager.Task::(_decref) Task=`469d8e52-c81e-4c7a-9a31-5967e7ff6472`::ref 0 aborting False >Thread-99::DEBUG::2014-08-11 14:42:38,780::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-99::DEBUG::2014-08-11 14:42:38,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0c56dff-7f94-4249-ad1e-bf577d704737`::moving from state init -> state preparing >Thread-99::INFO::2014-08-11 14:42:38,792::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-99::INFO::2014-08-11 14:42:38,811::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-99::DEBUG::2014-08-11 14:42:38,813::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0c56dff-7f94-4249-ad1e-bf577d704737`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-99::DEBUG::2014-08-11 14:42:38,814::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0c56dff-7f94-4249-ad1e-bf577d704737`::moving from state preparing -> state finished >Thread-99::DEBUG::2014-08-11 14:42:38,815::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-99::DEBUG::2014-08-11 14:42:38,816::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-99::DEBUG::2014-08-11 14:42:38,818::task::993::Storage.TaskManager.Task::(_decref) Task=`a0c56dff-7f94-4249-ad1e-bf577d704737`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:42:38,823::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:42:38,838::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:42:38,840::task::595::Storage.TaskManager.Task::(_updateState) Task=`90bb4da6-89c8-43e9-bef6-6e908f80cb7d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:42:38,843::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:42:38,846::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`34857201-2460-471f-ba49-740d274f00d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:42:38,847::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:42:38,849::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:42:38,850::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`34857201-2460-471f-ba49-740d274f00d1`::Granted request >Thread-34::DEBUG::2014-08-11 14:42:38,854::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`90bb4da6-89c8-43e9-bef6-6e908f80cb7d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:42:38,855::task::993::Storage.TaskManager.Task::(_decref) Task=`90bb4da6-89c8-43e9-bef6-6e908f80cb7d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:42:38,905::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861035622400', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:38,907::task::1191::Storage.TaskManager.Task::(prepare) Task=`90bb4da6-89c8-43e9-bef6-6e908f80cb7d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861035622400', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:38,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`90bb4da6-89c8-43e9-bef6-6e908f80cb7d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:42:38,910::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:42:38,911::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:42:38,913::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:42:38,915::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:42:38,917::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:42:38,918::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:42:38,921::task::993::Storage.TaskManager.Task::(_decref) Task=`90bb4da6-89c8-43e9-bef6-6e908f80cb7d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:42:39,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:39,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132262 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:41,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:41,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132601 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:41,828::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:41,828::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:41,829::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:41,829::__init__::232::IOProcess::(_processLogs) (5561) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:41,829::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5561) Queuing response >Thread-16::DEBUG::2014-08-11 14:42:41,830::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:42:41,848::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418325 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:41,849::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:42:41,850::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:41,850::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:41,850::__init__::232::IOProcess::(_processLogs) (5562) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:42:41,851::__init__::232::IOProcess::(_processLogs) (5562) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:41,852::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:41,852::__init__::232::IOProcess::(_processLogs) (5563) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:41,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:41,853::__init__::232::IOProcess::(_processLogs) (5564) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:41,854::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:41,854::__init__::232::IOProcess::(_processLogs) (5565) Queuing response >Dummy-33::DEBUG::2014-08-11 14:42:43,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:43,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132487 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:43,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:43,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:43,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:42:45,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:45,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131953 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:47,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:47,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131842 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:48,864::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:48,865::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:48,939::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:48,962::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:48,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`31e39e0c-03dd-444b-b148-584e7fae3c21`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:42:48,967::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:42:48,982::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:48,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`31e39e0c-03dd-444b-b148-584e7fae3c21`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:48,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`31e39e0c-03dd-444b-b148-584e7fae3c21`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:48,986::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:48,988::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:48,989::task::993::Storage.TaskManager.Task::(_decref) Task=`31e39e0c-03dd-444b-b148-584e7fae3c21`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:42:49,017::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:42:49,020::task::595::Storage.TaskManager.Task::(_updateState) Task=`4abbbdb3-b87d-45d6-a303-662a61ef17f6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:42:49,023::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:42:49,027::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f8c2c7b6-d600-4ce8-bbed-fc29ebb2cbf1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:42:49,029::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:42:49,031::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:42:49,032::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f8c2c7b6-d600-4ce8-bbed-fc29ebb2cbf1`::Granted request >Thread-34::DEBUG::2014-08-11 14:42:49,035::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4abbbdb3-b87d-45d6-a303-662a61ef17f6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:42:49,037::task::993::Storage.TaskManager.Task::(_decref) Task=`4abbbdb3-b87d-45d6-a303-662a61ef17f6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:42:49,084::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861026054144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:49,086::task::1191::Storage.TaskManager.Task::(prepare) Task=`4abbbdb3-b87d-45d6-a303-662a61ef17f6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861026054144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:49,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`4abbbdb3-b87d-45d6-a303-662a61ef17f6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:42:49,088::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:42:49,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:42:49,091::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:42:49,093::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:42:49,094::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:42:49,095::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:42:49,096::task::993::Storage.TaskManager.Task::(_decref) Task=`4abbbdb3-b87d-45d6-a303-662a61ef17f6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:42:49,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:49,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132357 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:51,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:51,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133921 s, 76.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:51,856::__init__::232::IOProcess::(_processLogs) (5566) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:42:51,857::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:42:51,879::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000583997 s, 779 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:42:51,881::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:42:51,881::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:51,882::__init__::232::IOProcess::(_processLogs) (5567) Queuing response >Thread-14::DEBUG::2014-08-11 14:42:51,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:51,886::__init__::232::IOProcess::(_processLogs) (5568) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:51,887::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:51,887::__init__::232::IOProcess::(_processLogs) (5569) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:51,888::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:42:51,888::__init__::232::IOProcess::(_processLogs) (5570) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:42:51,889::__init__::232::IOProcess::(_processLogs) (5570) Queuing response >Dummy-33::DEBUG::2014-08-11 14:42:53,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:53,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132345 s, 77.4 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:42:53,934::task::595::Storage.TaskManager.Task::(_updateState) Task=`32e3cf0e-8e8a-47b4-b329-f552f7eb7773`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:42:53,934::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:42:53,957::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000583997', 'lastCheck': '2.1', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:42:53,957::task::1191::Storage.TaskManager.Task::(prepare) Task=`32e3cf0e-8e8a-47b4-b329-f552f7eb7773`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000583997', 'lastCheck': '2.1', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:42:53,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`32e3cf0e-8e8a-47b4-b329-f552f7eb7773`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:53,957::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:53,958::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:53,958::task::993::Storage.TaskManager.Task::(_decref) Task=`32e3cf0e-8e8a-47b4-b329-f552f7eb7773`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:53,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:53,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:53,976::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:53,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:42:55,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:55,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132153 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:42:57,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:42:57,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133934 s, 76.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:58,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:58,985::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:42:59,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:42:59,110::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:42:59,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a2f555c-a2bc-4dcc-894c-eadf40a72d17`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:42:59,111::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:42:59,125::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:59,126::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a2f555c-a2bc-4dcc-894c-eadf40a72d17`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:42:59,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a2f555c-a2bc-4dcc-894c-eadf40a72d17`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:42:59,129::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:42:59,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:42:59,131::task::993::Storage.TaskManager.Task::(_decref) Task=`8a2f555c-a2bc-4dcc-894c-eadf40a72d17`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:42:59,146::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:42:59,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecd99f3f-4e4c-4f2a-a072-704e597d1c80`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:42:59,150::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:42:59,155::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7835bec0-c056-492a-a550-09875538b0c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:42:59,156::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:42:59,159::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:42:59,160::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7835bec0-c056-492a-a550-09875538b0c5`::Granted request >Thread-34::DEBUG::2014-08-11 14:42:59,164::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ecd99f3f-4e4c-4f2a-a072-704e597d1c80`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:42:59,165::task::993::Storage.TaskManager.Task::(_decref) Task=`ecd99f3f-4e4c-4f2a-a072-704e597d1c80`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:42:59,225::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861003051008', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:59,227::task::1191::Storage.TaskManager.Task::(prepare) Task=`ecd99f3f-4e4c-4f2a-a072-704e597d1c80`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1861003051008', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:42:59,228::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecd99f3f-4e4c-4f2a-a072-704e597d1c80`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:42:59,229::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:42:59,230::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:42:59,232::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:42:59,235::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:42:59,236::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:42:59,237::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:42:59,239::task::993::Storage.TaskManager.Task::(_decref) Task=`ecd99f3f-4e4c-4f2a-a072-704e597d1c80`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:42:59,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:00,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133286 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:01,891::__init__::232::IOProcess::(_processLogs) (5571) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:43:01,892::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:43:01,911::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417107 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:01,912::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:01,912::__init__::232::IOProcess::(_processLogs) (5572) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:43:01,913::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:01,914::__init__::232::IOProcess::(_processLogs) (5573) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:01,915::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:01,915::__init__::232::IOProcess::(_processLogs) (5574) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:01,916::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:01,916::__init__::232::IOProcess::(_processLogs) (5575) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:01,917::__init__::232::IOProcess::(_processLogs) (5575) Queuing response >Dummy-33::DEBUG::2014-08-11 14:43:02,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:02,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0234832 s, 43.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:04,065::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:04,067::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:04,158::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:04,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:04,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133184 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:06,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:06,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131879 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:08,377::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:08,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132189 s, 77.5 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:43:09,073::task::595::Storage.TaskManager.Task::(_updateState) Task=`8eb2183d-0bd1-4d51-8975-30a554e09bbf`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:09,073::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:43:09,158::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417107', 'lastCheck': '7.2', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:43:09,158::task::1191::Storage.TaskManager.Task::(prepare) Task=`8eb2183d-0bd1-4d51-8975-30a554e09bbf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417107', 'lastCheck': '7.2', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:43:09,158::task::595::Storage.TaskManager.Task::(_updateState) Task=`8eb2183d-0bd1-4d51-8975-30a554e09bbf`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:09,158::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:09,159::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:09,159::task::993::Storage.TaskManager.Task::(_decref) Task=`8eb2183d-0bd1-4d51-8975-30a554e09bbf`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:09,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:09,166::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:09,167::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:09,181::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:09,254::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:43:09,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c960e50-a030-4805-9900-a01321bf1e1f`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:09,255::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:43:09,268::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:09,268::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c960e50-a030-4805-9900-a01321bf1e1f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:09,268::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c960e50-a030-4805-9900-a01321bf1e1f`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:09,268::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:09,269::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:09,269::task::993::Storage.TaskManager.Task::(_decref) Task=`4c960e50-a030-4805-9900-a01321bf1e1f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:43:09,279::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:43:09,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`271ead50-a5f1-4b45-b163-a919f6a6aa12`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:43:09,280::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:43:09,281::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`787d71d0-294c-469a-96d6-e487600e9e39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:43:09,281::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:43:09,281::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:43:09,281::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`787d71d0-294c-469a-96d6-e487600e9e39`::Granted request >Thread-34::DEBUG::2014-08-11 14:43:09,282::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`271ead50-a5f1-4b45-b163-a919f6a6aa12`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:43:09,282::task::993::Storage.TaskManager.Task::(_decref) Task=`271ead50-a5f1-4b45-b163-a919f6a6aa12`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:43:09,303::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860969824256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:09,305::task::1191::Storage.TaskManager.Task::(prepare) Task=`271ead50-a5f1-4b45-b163-a919f6a6aa12`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860969824256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:09,306::task::595::Storage.TaskManager.Task::(_updateState) Task=`271ead50-a5f1-4b45-b163-a919f6a6aa12`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:43:09,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:43:09,309::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:43:09,312::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:43:09,314::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:43:09,315::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:43:09,317::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:43:09,320::task::993::Storage.TaskManager.Task::(_decref) Task=`271ead50-a5f1-4b45-b163-a919f6a6aa12`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:43:10,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:10,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013228 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:11,918::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:43:11,919::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:11,919::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:11,920::__init__::232::IOProcess::(_processLogs) (5576) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:43:11,920::__init__::232::IOProcess::(_processLogs) DEBUG|(5576) Queuing response >Thread-16::DEBUG::2014-08-11 14:43:11,921::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:43:11,939::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000429579 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:11,940::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:43:11,940::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:11,941::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:11,941::__init__::232::IOProcess::(_processLogs) (5577) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:43:11,942::__init__::232::IOProcess::(_processLogs) DEBUG|(5577) Queuing response >Thread-14::DEBUG::2014-08-11 14:43:11,943::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:43:11,943::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:11,944::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:11,944::__init__::232::IOProcess::(_processLogs) (5578) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:11,944::__init__::232::IOProcess::(_processLogs) DEBUG|(5578) Queuing response >Thread-14::DEBUG::2014-08-11 14:43:11,945::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:43:11,946::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:11,946::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:11,946::__init__::232::IOProcess::(_processLogs) (5579) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:11,947::__init__::232::IOProcess::(_processLogs) DEBUG|(5579) Queuing response >Thread-14::DEBUG::2014-08-11 14:43:11,947::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:43:11,948::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:11,948::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:11,948::__init__::232::IOProcess::(_processLogs) (5580) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:11,949::__init__::232::IOProcess::(_processLogs) DEBUG|(5580) Queuing response >Thread-55::DEBUG::2014-08-11 14:43:12,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`5883155d-8cdd-4970-b4bd-891c61791c19`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:43:12,450::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:43:12,450::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:43:12,451::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:12,451::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:12,451::__init__::232::IOProcess::(_processLogs) (5581) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:43:12,454::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:12,454::__init__::232::IOProcess::(_processLogs) (5582) Queuing response >Thread-55::INFO::2014-08-11 14:43:12,455::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:43:12,455::task::1191::Storage.TaskManager.Task::(prepare) Task=`5883155d-8cdd-4970-b4bd-891c61791c19`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:43:12,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`5883155d-8cdd-4970-b4bd-891c61791c19`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:43:12,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:43:12,456::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:43:12,456::task::993::Storage.TaskManager.Task::(_decref) Task=`5883155d-8cdd-4970-b4bd-891c61791c19`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:43:12,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:12,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0149476 s, 68.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:14,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:14,175::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:14,258::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:14,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:14,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132269 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:16,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:16,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013208 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:18,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:18,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134502 s, 76.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:19,265::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:19,267::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:19,337::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:43:19,360::task::595::Storage.TaskManager.Task::(_updateState) Task=`9472ec40-497c-4a66-9ff2-a9ba48eaab73`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:19,362::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:19,362::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:43:19,376::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:19,376::task::1191::Storage.TaskManager.Task::(prepare) Task=`9472ec40-497c-4a66-9ff2-a9ba48eaab73`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:19,376::task::595::Storage.TaskManager.Task::(_updateState) Task=`9472ec40-497c-4a66-9ff2-a9ba48eaab73`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:19,376::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:19,377::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:19,377::task::993::Storage.TaskManager.Task::(_decref) Task=`9472ec40-497c-4a66-9ff2-a9ba48eaab73`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:43:19,387::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:43:19,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`291dbba0-28a5-4455-9c4f-004b559a5591`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:43:19,388::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:43:19,389::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f2d3024a-1022-47e7-b4ef-bf8ca9b166b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:43:19,389::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:43:19,390::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:43:19,390::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f2d3024a-1022-47e7-b4ef-bf8ca9b166b2`::Granted request >Thread-34::DEBUG::2014-08-11 14:43:19,390::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`291dbba0-28a5-4455-9c4f-004b559a5591`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:43:19,391::task::993::Storage.TaskManager.Task::(_decref) Task=`291dbba0-28a5-4455-9c4f-004b559a5591`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:43:19,434::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860774658048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:19,436::task::1191::Storage.TaskManager.Task::(prepare) Task=`291dbba0-28a5-4455-9c4f-004b559a5591`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860774658048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:19,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`291dbba0-28a5-4455-9c4f-004b559a5591`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:43:19,438::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:43:19,440::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:43:19,442::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:43:19,445::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:43:19,446::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:43:19,448::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:43:19,450::task::993::Storage.TaskManager.Task::(_decref) Task=`291dbba0-28a5-4455-9c4f-004b559a5591`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:43:20,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:20,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013392 s, 76.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:21,951::__init__::232::IOProcess::(_processLogs) (5583) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:43:21,952::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:43:21,970::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423301 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:21,971::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:21,971::__init__::232::IOProcess::(_processLogs) (5584) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:43:21,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:21,973::__init__::232::IOProcess::(_processLogs) (5585) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:21,974::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:21,974::__init__::232::IOProcess::(_processLogs) (5586) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:21,975::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:21,975::__init__::232::IOProcess::(_processLogs) (5587) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:21,976::__init__::232::IOProcess::(_processLogs) (5587) Queuing response >Dummy-33::DEBUG::2014-08-11 14:43:22,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:22,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134852 s, 75.9 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:43:24,272::task::595::Storage.TaskManager.Task::(_updateState) Task=`82a7e549-c2fe-428f-a86f-243b0736b245`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:24,272::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:43:24,358::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423301', 'lastCheck': '2.4', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:43:24,358::task::1191::Storage.TaskManager.Task::(prepare) Task=`82a7e549-c2fe-428f-a86f-243b0736b245`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423301', 'lastCheck': '2.4', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:43:24,358::task::595::Storage.TaskManager.Task::(_updateState) Task=`82a7e549-c2fe-428f-a86f-243b0736b245`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:24,358::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:24,359::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:24,359::task::993::Storage.TaskManager.Task::(_decref) Task=`82a7e549-c2fe-428f-a86f-243b0736b245`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:24,370::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:24,371::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:24,372::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:24,378::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:24,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:24,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136134 s, 75.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:26,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:26,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0274511 s, 37.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:28,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:28,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133518 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:29,378::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:29,380::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:29,458::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:29,468::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:43:29,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`369c8d5c-a7eb-4fc9-a588-8090d931ae74`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:29,469::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:43:29,482::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:29,482::task::1191::Storage.TaskManager.Task::(prepare) Task=`369c8d5c-a7eb-4fc9-a588-8090d931ae74`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:29,482::task::595::Storage.TaskManager.Task::(_updateState) Task=`369c8d5c-a7eb-4fc9-a588-8090d931ae74`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:29,483::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:29,483::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:29,483::task::993::Storage.TaskManager.Task::(_decref) Task=`369c8d5c-a7eb-4fc9-a588-8090d931ae74`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:43:29,494::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:43:29,495::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6ab3fd1-6e20-44cf-915f-98396d42c135`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:43:29,495::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:43:29,496::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`efebff17-a79d-4e7e-bb4b-dd066cdc40bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:43:29,496::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:43:29,496::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:43:29,497::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`efebff17-a79d-4e7e-bb4b-dd066cdc40bc`::Granted request >Thread-34::DEBUG::2014-08-11 14:43:29,497::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c6ab3fd1-6e20-44cf-915f-98396d42c135`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:43:29,497::task::993::Storage.TaskManager.Task::(_decref) Task=`c6ab3fd1-6e20-44cf-915f-98396d42c135`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:43:29,517::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860731338752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:29,518::task::1191::Storage.TaskManager.Task::(prepare) Task=`c6ab3fd1-6e20-44cf-915f-98396d42c135`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860731338752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:29,518::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6ab3fd1-6e20-44cf-915f-98396d42c135`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:43:29,518::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:43:29,518::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:43:29,519::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:43:29,519::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:43:29,520::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:43:29,522::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:43:29,524::task::993::Storage.TaskManager.Task::(_decref) Task=`c6ab3fd1-6e20-44cf-915f-98396d42c135`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:43:31,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:31,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132488 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:31,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:31,981::__init__::232::IOProcess::(_processLogs) (5588) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:43:31,984::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:43:31,986::__init__::232::IOProcess::(_processLogs) (5588) Queuing response >Thread-16::DEBUG::2014-08-11 14:43:32,044::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00395789 s, 115 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:32,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:32,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:32,051::__init__::232::IOProcess::(_processLogs) (5589) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:43:32,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:32,054::__init__::232::IOProcess::(_processLogs) (5590) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:32,055::__init__::232::IOProcess::(_processLogs) (5590) Queuing response >Thread-14::DEBUG::2014-08-11 14:43:32,071::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:32,072::__init__::232::IOProcess::(_processLogs) (5591) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:32,075::__init__::232::IOProcess::(_processLogs) (5591) Queuing response >Thread-14::DEBUG::2014-08-11 14:43:32,077::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:43:32,079::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:32,081::__init__::232::IOProcess::(_processLogs) (5592) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:43:33,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:33,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131753 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:34,465::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:34,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:34,467::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:35,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:35,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131784 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:37,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:37,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132149 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:39,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:39,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132069 s, 77.5 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:43:39,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`2be7fc3a-f87d-4663-9fa1-2da2fc06f246`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:39,471::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:39,474::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 14:43:39,547::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:57368 >Detector thread::DEBUG::2014-08-11 14:43:39,562::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:57368 >Thread-97::INFO::2014-08-11 14:43:39,561::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00395789', 'lastCheck': '7.5', 'valid': True}} >Detector thread::DEBUG::2014-08-11 14:43:39,564::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:57368 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:39,565::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:39,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`2be7fc3a-f87d-4663-9fa1-2da2fc06f246`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00395789', 'lastCheck': '7.5', 'valid': True}} >Detector thread::DEBUG::2014-08-11 14:43:39,567::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 57368) >Thread-97::DEBUG::2014-08-11 14:43:39,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`2be7fc3a-f87d-4663-9fa1-2da2fc06f246`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:39,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:39,573::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:39,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:39,576::task::993::Storage.TaskManager.Task::(_decref) Task=`2be7fc3a-f87d-4663-9fa1-2da2fc06f246`::ref 0 aborting False >Thread-100::DEBUG::2014-08-11 14:43:39,577::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-100::DEBUG::2014-08-11 14:43:39,582::task::595::Storage.TaskManager.Task::(_updateState) Task=`0258b766-433a-416f-b367-3d80a3b60a1f`::moving from state init -> state preparing >Thread-100::INFO::2014-08-11 14:43:39,584::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-100::INFO::2014-08-11 14:43:39,598::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-100::DEBUG::2014-08-11 14:43:39,598::task::1191::Storage.TaskManager.Task::(prepare) Task=`0258b766-433a-416f-b367-3d80a3b60a1f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-100::DEBUG::2014-08-11 14:43:39,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`0258b766-433a-416f-b367-3d80a3b60a1f`::moving from state preparing -> state finished >Thread-100::DEBUG::2014-08-11 14:43:39,599::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-100::DEBUG::2014-08-11 14:43:39,599::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-100::DEBUG::2014-08-11 14:43:39,599::task::993::Storage.TaskManager.Task::(_decref) Task=`0258b766-433a-416f-b367-3d80a3b60a1f`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:43:39,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:43:39,613::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:43:39,615::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba561b61-691a-4fed-957e-309e8f580d05`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:43:39,617::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:43:39,621::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3bb3c692-de25-4f97-94e6-24e81d4b4f97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:43:39,622::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:43:39,624::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:43:39,625::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3bb3c692-de25-4f97-94e6-24e81d4b4f97`::Granted request >Thread-34::DEBUG::2014-08-11 14:43:39,628::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ba561b61-691a-4fed-957e-309e8f580d05`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:43:39,629::task::993::Storage.TaskManager.Task::(_decref) Task=`ba561b61-691a-4fed-957e-309e8f580d05`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:43:39,675::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860662525952', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:39,676::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba561b61-691a-4fed-957e-309e8f580d05`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860662525952', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:39,677::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba561b61-691a-4fed-957e-309e8f580d05`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:43:39,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:43:39,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:43:39,681::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:43:39,683::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:43:39,684::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:43:39,685::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:43:39,688::task::993::Storage.TaskManager.Task::(_decref) Task=`ba561b61-691a-4fed-957e-309e8f580d05`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:43:41,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:41,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132096 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:42,086::__init__::232::IOProcess::(_processLogs) (5593) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:43:42,087::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:43:42,106::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416041 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:42,107::__init__::232::IOProcess::(_processLogs) (5594) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:43:42,108::__init__::232::IOProcess::(_processLogs) (5595) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:42,109::__init__::232::IOProcess::(_processLogs) (5596) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:42,110::__init__::232::IOProcess::(_processLogs) (5597) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:43:43,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:43,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131809 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:44,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:44,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:44,660::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:45,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:45,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132182 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:47,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:47,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135029 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:49,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:49,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013491 s, 75.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:49,664::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:49,665::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:49,708::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:43:49,762::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbbef5fe-be83-47ed-b001-0486fa79d84c`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:49,764::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:49,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:43:49,783::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:49,785::task::1191::Storage.TaskManager.Task::(prepare) Task=`cbbef5fe-be83-47ed-b001-0486fa79d84c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:49,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbbef5fe-be83-47ed-b001-0486fa79d84c`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:49,787::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:49,788::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:49,789::task::993::Storage.TaskManager.Task::(_decref) Task=`cbbef5fe-be83-47ed-b001-0486fa79d84c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:43:49,803::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:43:49,806::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9c4e0fc-5d50-4be9-b528-49cb08f52422`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:43:49,807::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:43:49,811::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aa635882-26ac-47f4-8f2d-0464afbe9218`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:43:49,813::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:43:49,815::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:43:49,817::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aa635882-26ac-47f4-8f2d-0464afbe9218`::Granted request >Thread-34::DEBUG::2014-08-11 14:43:49,819::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d9c4e0fc-5d50-4be9-b528-49cb08f52422`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:43:49,821::task::993::Storage.TaskManager.Task::(_decref) Task=`d9c4e0fc-5d50-4be9-b528-49cb08f52422`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:43:49,878::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860597448704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:49,880::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9c4e0fc-5d50-4be9-b528-49cb08f52422`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860597448704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:43:49,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9c4e0fc-5d50-4be9-b528-49cb08f52422`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:43:49,883::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:43:49,884::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:43:49,885::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:43:49,887::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:43:49,887::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:43:49,889::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:43:49,890::task::993::Storage.TaskManager.Task::(_decref) Task=`d9c4e0fc-5d50-4be9-b528-49cb08f52422`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:43:51,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:51,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132996 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:52,112::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:52,113::__init__::232::IOProcess::(_processLogs) (5598) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:43:52,114::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:43:52,131::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041771 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:43:52,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:52,133::__init__::232::IOProcess::(_processLogs) (5599) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:43:52,134::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:52,134::__init__::232::IOProcess::(_processLogs) (5600) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:43:52,135::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:52,136::__init__::232::IOProcess::(_processLogs) (5601) Queuing response >Thread-14::DEBUG::2014-08-11 14:43:52,137::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:43:52,137::__init__::232::IOProcess::(_processLogs) (5602) Queuing response >Dummy-33::DEBUG::2014-08-11 14:43:53,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:53,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132836 s, 77.1 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:43:54,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b3f6907-32b6-483b-ad57-ff434f819a66`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:54,690::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:43:54,761::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041771', 'lastCheck': '2.6', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:43:54,761::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b3f6907-32b6-483b-ad57-ff434f819a66`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041771', 'lastCheck': '2.6', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:43:54,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b3f6907-32b6-483b-ad57-ff434f819a66`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:54,761::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:54,762::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:54,762::task::993::Storage.TaskManager.Task::(_decref) Task=`7b3f6907-32b6-483b-ad57-ff434f819a66`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:54,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:54,776::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:54,777::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:43:54,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:55,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:55,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133105 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:43:57,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:43:57,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131798 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:59,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:59,784::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:59,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:43:59,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:43:59,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132669 s, 77.2 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:43:59,907::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:43:59,910::task::595::Storage.TaskManager.Task::(_updateState) Task=`81214cc8-913b-4c08-b203-179dad582157`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:43:59,911::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:43:59,925::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:59,928::task::1191::Storage.TaskManager.Task::(prepare) Task=`81214cc8-913b-4c08-b203-179dad582157`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:43:59,929::task::595::Storage.TaskManager.Task::(_updateState) Task=`81214cc8-913b-4c08-b203-179dad582157`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:43:59,930::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:43:59,931::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:43:59,932::task::993::Storage.TaskManager.Task::(_decref) Task=`81214cc8-913b-4c08-b203-179dad582157`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:43:59,952::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:43:59,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`e828ef37-3dc6-49a2-9ded-39e10218d72b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:43:59,956::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:43:59,959::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`292a2b72-b8ee-41af-8e1a-89f5685b1ea1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:43:59,960::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:43:59,962::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:43:59,963::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`292a2b72-b8ee-41af-8e1a-89f5685b1ea1`::Granted request >Thread-34::DEBUG::2014-08-11 14:43:59,966::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e828ef37-3dc6-49a2-9ded-39e10218d72b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:43:59,968::task::993::Storage.TaskManager.Task::(_decref) Task=`e828ef37-3dc6-49a2-9ded-39e10218d72b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:44:00,021::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860585652224', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:00,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`e828ef37-3dc6-49a2-9ded-39e10218d72b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860585652224', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:00,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`e828ef37-3dc6-49a2-9ded-39e10218d72b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:44:00,024::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:44:00,025::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:44:00,027::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:44:00,029::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:44:00,030::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:44:00,031::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:44:00,034::task::993::Storage.TaskManager.Task::(_decref) Task=`e828ef37-3dc6-49a2-9ded-39e10218d72b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:44:01,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:01,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0386818 s, 26.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:02,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:02,148::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:02,152::__init__::232::IOProcess::(_processLogs) (5603) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:44:02,160::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:44:02,186::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0013147 s, 346 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:02,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:02,195::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:02,196::__init__::232::IOProcess::(_processLogs) (5604) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:02,198::__init__::232::IOProcess::(_processLogs) (5604) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:02,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:02,207::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:02,210::__init__::232::IOProcess::(_processLogs) (5605) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:02,213::__init__::232::IOProcess::(_processLogs) (5605) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:02,217::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:02,219::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:02,220::__init__::232::IOProcess::(_processLogs) (5606) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:02,224::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:02,227::__init__::232::IOProcess::(_processLogs) (5607) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:02,230::__init__::232::IOProcess::(_processLogs) (5607) Queuing response >Dummy-33::DEBUG::2014-08-11 14:44:04,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:04,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132552 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:04,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:04,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:04,890::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:44:06,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:06,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013181 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:08,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:08,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132223 s, 77.4 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:44:09,893::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d0615bc-63a6-40e7-880c-bbe1244ebd2a`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:44:09,893::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:09,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:44:09,963::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0013147', 'lastCheck': '7.7', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:44:09,964::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d0615bc-63a6-40e7-880c-bbe1244ebd2a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0013147', 'lastCheck': '7.7', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:44:09,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d0615bc-63a6-40e7-880c-bbe1244ebd2a`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:09,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:09,967::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:09,968::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:44:09,970::task::993::Storage.TaskManager.Task::(_decref) Task=`4d0615bc-63a6-40e7-880c-bbe1244ebd2a`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:09,972::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:09,993::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:10,058::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:44:10,058::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f9101f1-2fb1-47fa-a1a3-65ab5ac6eebc`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:44:10,059::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:44:10,071::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:10,072::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f9101f1-2fb1-47fa-a1a3-65ab5ac6eebc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:10,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f9101f1-2fb1-47fa-a1a3-65ab5ac6eebc`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:44:10,072::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:10,072::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:44:10,072::task::993::Storage.TaskManager.Task::(_decref) Task=`0f9101f1-2fb1-47fa-a1a3-65ab5ac6eebc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:44:10,083::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:44:10,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`51322133-dde3-4255-80a9-946742454bf8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:44:10,084::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:44:10,084::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7abe46d3-f8a6-4c82-aea3-db4fd2fb43a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:44:10,085::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:44:10,085::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:44:10,085::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7abe46d3-f8a6-4c82-aea3-db4fd2fb43a9`::Granted request >Thread-34::DEBUG::2014-08-11 14:44:10,086::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`51322133-dde3-4255-80a9-946742454bf8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:44:10,086::task::993::Storage.TaskManager.Task::(_decref) Task=`51322133-dde3-4255-80a9-946742454bf8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:44:10,121::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860585390080', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:10,122::task::1191::Storage.TaskManager.Task::(prepare) Task=`51322133-dde3-4255-80a9-946742454bf8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860585390080', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:10,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`51322133-dde3-4255-80a9-946742454bf8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:44:10,126::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:44:10,127::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:44:10,129::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:44:10,131::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:44:10,132::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:44:10,133::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:44:10,135::task::993::Storage.TaskManager.Task::(_decref) Task=`51322133-dde3-4255-80a9-946742454bf8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:44:10,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:10,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132724 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:12,229::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:12,231::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:12,231::__init__::232::IOProcess::(_processLogs) (5608) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:44:12,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:44:12,251::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425634 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:12,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:12,253::__init__::232::IOProcess::(_processLogs) (5609) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:12,254::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:12,255::__init__::232::IOProcess::(_processLogs) (5610) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:12,255::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5610) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:12,256::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:12,256::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:12,256::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:12,256::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5611) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:12,257::__init__::232::IOProcess::(_processLogs) (5611) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:12,257::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:12,258::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:12,258::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:12,258::__init__::232::IOProcess::(_processLogs) (5612) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:12,258::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5612) Queuing response >Dummy-33::DEBUG::2014-08-11 14:44:12,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:12,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134168 s, 76.3 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:44:12,590::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a8b9260-13c6-4b36-bbf5-f75180fe5ae3`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:44:12,592::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:44:12,594::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:12,595::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:12,595::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:12,595::__init__::232::IOProcess::(_processLogs) (5613) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:44:12,596::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5613) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:12,598::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:12,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:12,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:12,599::__init__::232::IOProcess::(_processLogs) (5614) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:44:12,599::__init__::232::IOProcess::(_processLogs) (5614) Queuing response >Thread-55::INFO::2014-08-11 14:44:12,600::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:44:12,600::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a8b9260-13c6-4b36-bbf5-f75180fe5ae3`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:44:12,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a8b9260-13c6-4b36-bbf5-f75180fe5ae3`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:44:12,601::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:44:12,601::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:44:12,601::task::993::Storage.TaskManager.Task::(_decref) Task=`7a8b9260-13c6-4b36-bbf5-f75180fe5ae3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:44:14,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:14,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132058 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:14,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:14,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:15,062::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:44:16,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:16,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132194 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:18,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:18,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133263 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:20,069::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:20,070::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:20,151::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:44:20,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`62b4d2b2-14a7-4bb5-9cbe-cc74ad25a2d9`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:44:20,165::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:20,166::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:44:20,180::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:20,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`62b4d2b2-14a7-4bb5-9cbe-cc74ad25a2d9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:20,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`62b4d2b2-14a7-4bb5-9cbe-cc74ad25a2d9`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:44:20,180::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:20,181::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:44:20,181::task::993::Storage.TaskManager.Task::(_decref) Task=`62b4d2b2-14a7-4bb5-9cbe-cc74ad25a2d9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:44:20,194::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:44:20,195::task::595::Storage.TaskManager.Task::(_updateState) Task=`0733b2e7-8983-4afa-839f-926ffbe14d6c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:44:20,195::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:44:20,196::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bc6a2782-33ca-4d42-96ca-8bfa60481f6a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:44:20,196::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:44:20,197::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:44:20,197::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bc6a2782-33ca-4d42-96ca-8bfa60481f6a`::Granted request >Thread-34::DEBUG::2014-08-11 14:44:20,197::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0733b2e7-8983-4afa-839f-926ffbe14d6c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:44:20,198::task::993::Storage.TaskManager.Task::(_decref) Task=`0733b2e7-8983-4afa-839f-926ffbe14d6c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:44:20,218::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860504846336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:20,220::task::1191::Storage.TaskManager.Task::(prepare) Task=`0733b2e7-8983-4afa-839f-926ffbe14d6c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860504846336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:20,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`0733b2e7-8983-4afa-839f-926ffbe14d6c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:44:20,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:44:20,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:44:20,226::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:44:20,227::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:44:20,228::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:44:20,229::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:44:20,232::task::993::Storage.TaskManager.Task::(_decref) Task=`0733b2e7-8983-4afa-839f-926ffbe14d6c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:44:20,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:20,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132728 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:22,260::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:22,261::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:22,261::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:22,261::__init__::232::IOProcess::(_processLogs) (5615) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:22,262::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5615) Queuing response >Thread-16::DEBUG::2014-08-11 14:44:22,263::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:44:22,283::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000441046 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:22,285::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:22,287::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:22,289::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:22,291::__init__::232::IOProcess::(_processLogs) (5616) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:22,294::__init__::232::IOProcess::(_processLogs) (5616) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:22,296::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:22,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:22,299::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:22,301::__init__::232::IOProcess::(_processLogs) (5617) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:22,302::__init__::232::IOProcess::(_processLogs) (5617) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:22,303::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:22,303::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:22,304::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:22,304::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5618) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:22,305::__init__::232::IOProcess::(_processLogs) (5618) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:22,305::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:22,306::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:22,306::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:22,307::__init__::232::IOProcess::(_processLogs) (5619) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:22,307::__init__::232::IOProcess::(_processLogs) (5619) Queuing response >Dummy-33::DEBUG::2014-08-11 14:44:22,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:22,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212248 s, 48.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:24,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:24,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131833 s, 77.7 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:44:25,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee555f3b-1e9f-47db-b803-de8cce8d20c2`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:44:25,084::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:44:25,162::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000441046', 'lastCheck': '2.9', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:44:25,162::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee555f3b-1e9f-47db-b803-de8cce8d20c2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000441046', 'lastCheck': '2.9', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:44:25,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee555f3b-1e9f-47db-b803-de8cce8d20c2`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:44:25,162::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:25,163::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:44:25,163::task::993::Storage.TaskManager.Task::(_decref) Task=`ee555f3b-1e9f-47db-b803-de8cce8d20c2`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:25,174::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:25,175::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:25,176::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:25,183::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:44:26,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:26,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131922 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:28,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:28,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132569 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:30,181::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:30,182::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:30,247::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:44:30,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`09aebd5f-d108-4203-bff5-7ae30e9d7f74`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:30,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:44:30,266::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:44:30,280::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:30,281::task::1191::Storage.TaskManager.Task::(prepare) Task=`09aebd5f-d108-4203-bff5-7ae30e9d7f74`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:30,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`09aebd5f-d108-4203-bff5-7ae30e9d7f74`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:44:30,281::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:30,281::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:44:30,282::task::993::Storage.TaskManager.Task::(_decref) Task=`09aebd5f-d108-4203-bff5-7ae30e9d7f74`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:44:30,293::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:44:30,294::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5703d9c-8aec-4dcc-9edd-4bb9567fc91f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:44:30,294::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:44:30,295::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6cb319fe-6f1b-4561-b452-8719b335755c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:44:30,295::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:44:30,295::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:44:30,296::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6cb319fe-6f1b-4561-b452-8719b335755c`::Granted request >Thread-34::DEBUG::2014-08-11 14:44:30,296::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5703d9c-8aec-4dcc-9edd-4bb9567fc91f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:44:30,296::task::993::Storage.TaskManager.Task::(_decref) Task=`d5703d9c-8aec-4dcc-9edd-4bb9567fc91f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:44:30,316::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860504780800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:30,317::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5703d9c-8aec-4dcc-9edd-4bb9567fc91f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860504780800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:30,317::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5703d9c-8aec-4dcc-9edd-4bb9567fc91f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:44:30,317::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:44:30,317::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:44:30,318::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:44:30,318::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:44:30,318::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:44:30,319::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:44:30,319::task::993::Storage.TaskManager.Task::(_decref) Task=`d5703d9c-8aec-4dcc-9edd-4bb9567fc91f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:44:30,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:30,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133529 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:32,309::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:32,311::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:32,312::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:32,314::__init__::232::IOProcess::(_processLogs) (5620) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:32,316::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5620) Queuing response >Thread-16::DEBUG::2014-08-11 14:44:32,317::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:44:32,341::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00358687 s, 127 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:32,344::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:32,345::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:32,347::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:32,348::__init__::232::IOProcess::(_processLogs) (5621) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:32,350::__init__::232::IOProcess::(_processLogs) (5621) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:32,352::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:32,353::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:32,354::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:32,356::__init__::232::IOProcess::(_processLogs) (5622) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:32,357::__init__::232::IOProcess::(_processLogs) (5622) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:32,362::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:32,364::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:32,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:32,370::__init__::232::IOProcess::(_processLogs) (5623) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:32,372::__init__::232::IOProcess::(_processLogs) (5623) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:32,377::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:32,381::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:32,383::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:32,384::__init__::232::IOProcess::(_processLogs) (5624) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:32,385::__init__::232::IOProcess::(_processLogs) (5624) Queuing response >Dummy-33::DEBUG::2014-08-11 14:44:33,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:33,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131946 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:35,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:35,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133141 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:35,275::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:35,276::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:35,362::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:44:37,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:37,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132226 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:39,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:39,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132506 s, 77.3 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:44:40,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdd3681a-6822-4375-ac82-aee87b3102b1`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:44:40,281::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Detector thread::DEBUG::2014-08-11 14:44:40,353::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:35713 >Thread-97::INFO::2014-08-11 14:44:40,362::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00358687', 'lastCheck': '8.0', 'valid': True}} >Detector thread::DEBUG::2014-08-11 14:44:40,363::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:35713 >Thread-97::DEBUG::2014-08-11 14:44:40,364::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdd3681a-6822-4375-ac82-aee87b3102b1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00358687', 'lastCheck': '8.0', 'valid': True}} >Detector thread::DEBUG::2014-08-11 14:44:40,365::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:35713 >Thread-97::DEBUG::2014-08-11 14:44:40,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdd3681a-6822-4375-ac82-aee87b3102b1`::moving from state preparing -> state finished >Detector thread::DEBUG::2014-08-11 14:44:40,366::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 35713) >Thread-97::DEBUG::2014-08-11 14:44:40,367::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:40,371::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:40,372::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:40,374::task::993::Storage.TaskManager.Task::(_decref) Task=`bdd3681a-6822-4375-ac82-aee87b3102b1`::ref 0 aborting False >Thread-101::DEBUG::2014-08-11 14:44:40,376::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:40,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-101::DEBUG::2014-08-11 14:44:40,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`49e86152-245b-4b01-888a-1bd0ab3d491e`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:40,387::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-101::INFO::2014-08-11 14:44:40,390::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-101::INFO::2014-08-11 14:44:40,405::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-101::DEBUG::2014-08-11 14:44:40,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`49e86152-245b-4b01-888a-1bd0ab3d491e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-101::DEBUG::2014-08-11 14:44:40,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`49e86152-245b-4b01-888a-1bd0ab3d491e`::moving from state preparing -> state finished >Thread-101::DEBUG::2014-08-11 14:44:40,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-101::DEBUG::2014-08-11 14:44:40,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-101::DEBUG::2014-08-11 14:44:40,407::task::993::Storage.TaskManager.Task::(_decref) Task=`49e86152-245b-4b01-888a-1bd0ab3d491e`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:44:40,415::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:44:40,425::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:44:40,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ce90d4d-aadc-46cd-8d79-82643651cddb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:44:40,434::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:44:40,441::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`195688f7-0336-46a1-9f78-3f798c01230a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:44:40,442::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:44:40,445::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:44:40,446::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`195688f7-0336-46a1-9f78-3f798c01230a`::Granted request >Thread-34::DEBUG::2014-08-11 14:44:40,449::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1ce90d4d-aadc-46cd-8d79-82643651cddb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:44:40,451::task::993::Storage.TaskManager.Task::(_decref) Task=`1ce90d4d-aadc-46cd-8d79-82643651cddb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:44:40,503::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860504780800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:40,504::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ce90d4d-aadc-46cd-8d79-82643651cddb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860504780800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:40,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ce90d4d-aadc-46cd-8d79-82643651cddb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:44:40,507::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:44:40,508::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:44:40,510::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:44:40,511::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:44:40,512::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:44:40,513::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:44:40,515::task::993::Storage.TaskManager.Task::(_decref) Task=`1ce90d4d-aadc-46cd-8d79-82643651cddb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:44:41,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:41,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013185 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:42,383::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:42,384::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:42,384::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:42,384::__init__::232::IOProcess::(_processLogs) (5625) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:42,385::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5625) Queuing response >Thread-16::DEBUG::2014-08-11 14:44:42,386::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:44:42,407::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00172286 s, 264 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:42,409::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:42,409::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:42,409::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:42,409::__init__::232::IOProcess::(_processLogs) (5626) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:42,411::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5626) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:42,411::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:42,412::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:42,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:42,413::__init__::232::IOProcess::(_processLogs) (5627) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:42,413::__init__::232::IOProcess::(_processLogs) (5627) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:42,413::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:42,414::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:42,415::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:42,415::__init__::232::IOProcess::(_processLogs) (5628) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:42,415::__init__::232::IOProcess::(_processLogs) (5628) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:42,415::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:42,416::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:42,416::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:42,417::__init__::232::IOProcess::(_processLogs) (5629) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:42,418::__init__::232::IOProcess::(_processLogs) (5629) Queuing response >Dummy-33::DEBUG::2014-08-11 14:44:43,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:43,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134973 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:45,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:45,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134003 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:45,399::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:45,404::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:45,408::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:44:47,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:47,451::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132228 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:49,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:49,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179749 s, 57.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:50,418::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:50,420::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:50,462::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:50,532::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-97::DEBUG::2014-08-11 14:44:50,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cd0671f-7cfb-4114-acb7-0131779cdb9a`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:44:50,533::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:44:50,546::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:50,546::task::1191::Storage.TaskManager.Task::(prepare) Task=`1cd0671f-7cfb-4114-acb7-0131779cdb9a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:44:50,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cd0671f-7cfb-4114-acb7-0131779cdb9a`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:44:50,547::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:50,547::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:44:50,547::task::993::Storage.TaskManager.Task::(_decref) Task=`1cd0671f-7cfb-4114-acb7-0131779cdb9a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:44:50,557::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 14:44:50,558::task::595::Storage.TaskManager.Task::(_updateState) Task=`18c02f77-7101-4776-bd6f-cc29e000ea63`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:44:50,558::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:44:50,559::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7b5620fc-411c-49db-a9c7-76aa755119cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:44:50,559::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:44:50,560::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:44:50,560::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7b5620fc-411c-49db-a9c7-76aa755119cf`::Granted request >Thread-34::DEBUG::2014-08-11 14:44:50,561::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`18c02f77-7101-4776-bd6f-cc29e000ea63`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:44:50,561::task::993::Storage.TaskManager.Task::(_decref) Task=`18c02f77-7101-4776-bd6f-cc29e000ea63`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:44:50,585::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479680512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:50,587::task::1191::Storage.TaskManager.Task::(prepare) Task=`18c02f77-7101-4776-bd6f-cc29e000ea63`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479680512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:44:50,588::task::595::Storage.TaskManager.Task::(_updateState) Task=`18c02f77-7101-4776-bd6f-cc29e000ea63`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:44:50,589::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:44:50,590::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:44:50,592::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:44:50,594::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:44:50,595::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:44:50,596::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:44:50,598::task::993::Storage.TaskManager.Task::(_decref) Task=`18c02f77-7101-4776-bd6f-cc29e000ea63`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:44:51,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:51,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133105 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:52,418::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:52,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:52,420::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:52,421::__init__::232::IOProcess::(_processLogs) (5630) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:44:52,421::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:44:52,422::__init__::232::IOProcess::(_processLogs) (5630) Queuing response >Thread-16::DEBUG::2014-08-11 14:44:52,440::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042416 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:44:52,441::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:52,442::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:52,442::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:52,442::__init__::232::IOProcess::(_processLogs) (5631) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:44:52,443::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5631) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:52,444::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:52,445::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:52,446::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:52,446::__init__::232::IOProcess::(_processLogs) (5632) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:52,446::__init__::232::IOProcess::(_processLogs) (5632) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:52,446::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:52,447::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:52,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:52,448::__init__::232::IOProcess::(_processLogs) (5633) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:52,448::__init__::232::IOProcess::(_processLogs) (5633) Queuing response >Thread-14::DEBUG::2014-08-11 14:44:52,448::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:44:52,449::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:44:52,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:44:52,450::__init__::232::IOProcess::(_processLogs) (5634) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:44:52,451::__init__::232::IOProcess::(_processLogs) (5634) Queuing response >Dummy-33::DEBUG::2014-08-11 14:44:53,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:53,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131743 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:55,469::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:55,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:55,530::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f043792-ec4e-4fec-b70a-8ca03d597dae`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:44:55,564::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:44:55,566::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042416', 'lastCheck': '3.1', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:44:55,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f043792-ec4e-4fec-b70a-8ca03d597dae`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042416', 'lastCheck': '3.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:44:55,567::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:44:55,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f043792-ec4e-4fec-b70a-8ca03d597dae`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:44:55,570::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:44:55,571::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:44:55,571::task::993::Storage.TaskManager.Task::(_decref) Task=`4f043792-ec4e-4fec-b70a-8ca03d597dae`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:44:55,591::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:44:55,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:55,674::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132226 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:57,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:57,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141888 s, 72.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:44:59,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:44:59,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131538 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:00,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:00,575::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:00,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:00,800::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:45:00,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`81797acc-afdd-47b5-900f-818b0e113778`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:45:00,801::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:45:00,814::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:00,815::task::1191::Storage.TaskManager.Task::(prepare) Task=`81797acc-afdd-47b5-900f-818b0e113778`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:00,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`81797acc-afdd-47b5-900f-818b0e113778`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:45:00,815::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:45:00,815::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:45:00,816::task::993::Storage.TaskManager.Task::(_decref) Task=`81797acc-afdd-47b5-900f-818b0e113778`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:45:00,838::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:45:00,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`54d6326d-22ac-47d0-9d4e-0c28080c35d6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:45:00,839::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:45:00,840::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`32cbdadf-aeb3-4e24-80c3-6dfdf859ebab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:45:00,840::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:45:00,841::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:45:00,841::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`32cbdadf-aeb3-4e24-80c3-6dfdf859ebab`::Granted request >Thread-34::DEBUG::2014-08-11 14:45:00,842::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`54d6326d-22ac-47d0-9d4e-0c28080c35d6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:45:00,842::task::993::Storage.TaskManager.Task::(_decref) Task=`54d6326d-22ac-47d0-9d4e-0c28080c35d6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:45:00,862::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479680512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:00,862::task::1191::Storage.TaskManager.Task::(prepare) Task=`54d6326d-22ac-47d0-9d4e-0c28080c35d6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479680512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:00,862::task::595::Storage.TaskManager.Task::(_updateState) Task=`54d6326d-22ac-47d0-9d4e-0c28080c35d6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:45:00,863::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:45:00,863::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:45:00,863::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:45:00,864::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:45:00,864::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:45:00,864::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:45:00,864::task::993::Storage.TaskManager.Task::(_decref) Task=`54d6326d-22ac-47d0-9d4e-0c28080c35d6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:45:01,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:01,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134525 s, 76.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:02,453::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:02,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:02,455::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:02,457::__init__::232::IOProcess::(_processLogs) (5635) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:02,458::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5635) Queuing response >Thread-16::DEBUG::2014-08-11 14:45:02,460::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:45:02,480::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000398592 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:02,483::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:02,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:02,486::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:02,487::__init__::232::IOProcess::(_processLogs) (5636) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:02,489::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5636) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:02,492::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:02,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:02,495::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:02,496::__init__::232::IOProcess::(_processLogs) (5637) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:02,497::__init__::232::IOProcess::(_processLogs) (5637) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:02,500::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:02,501::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:02,502::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:02,503::__init__::232::IOProcess::(_processLogs) (5638) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:02,505::__init__::232::IOProcess::(_processLogs) (5638) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:02,507::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:02,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:02,510::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:02,511::__init__::232::IOProcess::(_processLogs) (5639) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:02,512::__init__::232::IOProcess::(_processLogs) (5639) Queuing response >Dummy-33::DEBUG::2014-08-11 14:45:03,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:03,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148007 s, 69.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:05,666::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:05,667::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:05,763::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:45:05,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:05,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133025 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:07,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:08,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132307 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:10,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:10,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132084 s, 77.5 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:45:10,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c509332-856b-4d67-90e7-b543ce7b64a0`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:45:10,689::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:45:10,763::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398592', 'lastCheck': '8.3', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:45:10,765::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c509332-856b-4d67-90e7-b543ce7b64a0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398592', 'lastCheck': '8.3', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:45:10,766::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c509332-856b-4d67-90e7-b543ce7b64a0`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:45:10,768::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:10,771::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:10,772::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:10,777::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:10,776::task::993::Storage.TaskManager.Task::(_decref) Task=`2c509332-856b-4d67-90e7-b543ce7b64a0`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:10,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:10,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:10,886::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:45:10,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bf4a550-9b57-4ce2-ad54-05e4cbcde79b`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:45:10,887::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:45:10,901::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:10,901::task::1191::Storage.TaskManager.Task::(prepare) Task=`6bf4a550-9b57-4ce2-ad54-05e4cbcde79b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:10,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bf4a550-9b57-4ce2-ad54-05e4cbcde79b`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:45:10,902::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:45:10,902::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:45:10,902::task::993::Storage.TaskManager.Task::(_decref) Task=`6bf4a550-9b57-4ce2-ad54-05e4cbcde79b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:45:10,913::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:45:10,913::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b713f37-5892-4027-9bfd-76c60f9901eb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:45:10,914::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:45:10,914::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7ad88dd-a7f5-4ec3-9d9a-ce35f3215d52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:45:10,915::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:45:10,915::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:45:10,915::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7ad88dd-a7f5-4ec3-9d9a-ce35f3215d52`::Granted request >Thread-34::DEBUG::2014-08-11 14:45:10,916::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8b713f37-5892-4027-9bfd-76c60f9901eb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:45:10,916::task::993::Storage.TaskManager.Task::(_decref) Task=`8b713f37-5892-4027-9bfd-76c60f9901eb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:45:10,936::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479614976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:10,936::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b713f37-5892-4027-9bfd-76c60f9901eb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479614976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:10,937::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b713f37-5892-4027-9bfd-76c60f9901eb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:45:10,937::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:45:10,937::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:45:10,937::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:45:10,938::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:45:10,938::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:45:10,938::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:45:10,938::task::993::Storage.TaskManager.Task::(_decref) Task=`8b713f37-5892-4027-9bfd-76c60f9901eb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:45:12,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:12,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134237 s, 76.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:12,509::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:12,510::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:12,510::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:12,510::__init__::232::IOProcess::(_processLogs) (5640) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:12,511::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5640) Queuing response >Thread-16::DEBUG::2014-08-11 14:45:12,512::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:45:12,531::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416233 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:12,532::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:12,533::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:12,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:12,533::__init__::232::IOProcess::(_processLogs) (5641) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:12,534::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5641) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:12,535::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:12,536::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:12,536::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:12,536::__init__::232::IOProcess::(_processLogs) (5642) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:12,537::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5642) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:12,537::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:12,538::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:12,539::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:12,539::__init__::232::IOProcess::(_processLogs) (5643) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:12,539::__init__::232::IOProcess::(_processLogs) (5643) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:12,539::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:12,540::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:12,540::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:12,540::__init__::232::IOProcess::(_processLogs) (5644) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:12,541::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5644) Queuing response >Thread-55::DEBUG::2014-08-11 14:45:12,742::task::595::Storage.TaskManager.Task::(_updateState) Task=`a59cb738-9fb2-4e19-a561-3d87a4cb5e84`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:45:12,742::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:45:12,743::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:12,743::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:12,744::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:12,744::__init__::232::IOProcess::(_processLogs) (5645) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:45:12,745::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5645) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:12,746::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:12,746::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:12,746::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:12,746::__init__::232::IOProcess::(_processLogs) (5646) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:45:12,747::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5646) Queuing response >Thread-55::INFO::2014-08-11 14:45:12,747::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:45:12,748::task::1191::Storage.TaskManager.Task::(prepare) Task=`a59cb738-9fb2-4e19-a561-3d87a4cb5e84`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:45:12,748::task::595::Storage.TaskManager.Task::(_updateState) Task=`a59cb738-9fb2-4e19-a561-3d87a4cb5e84`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:45:12,749::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:45:12,749::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:45:12,749::task::993::Storage.TaskManager.Task::(_decref) Task=`a59cb738-9fb2-4e19-a561-3d87a4cb5e84`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:45:14,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:14,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132332 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:15,797::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:15,798::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:15,863::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:45:16,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:16,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134264 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:18,253::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:18,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136148 s, 75.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:20,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:20,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133092 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:20,865::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:20,866::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:20,952::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:20,966::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:20,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4786a44-0334-4efc-b78b-8ceef52f9a30`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:45:20,967::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:45:20,981::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:20,981::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4786a44-0334-4efc-b78b-8ceef52f9a30`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:20,982::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4786a44-0334-4efc-b78b-8ceef52f9a30`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:45:20,982::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:45:20,982::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:45:20,982::task::993::Storage.TaskManager.Task::(_decref) Task=`d4786a44-0334-4efc-b78b-8ceef52f9a30`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:45:20,993::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:45:20,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3157c16-ee89-455a-a33c-26012078549f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:45:20,994::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:45:20,994::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0c6a2574-505e-43c6-9d24-006f873948ee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:45:20,994::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:45:20,995::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:45:20,995::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0c6a2574-505e-43c6-9d24-006f873948ee`::Granted request >Thread-34::DEBUG::2014-08-11 14:45:20,996::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f3157c16-ee89-455a-a33c-26012078549f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:45:20,996::task::993::Storage.TaskManager.Task::(_decref) Task=`f3157c16-ee89-455a-a33c-26012078549f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:45:21,016::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479614976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:21,016::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3157c16-ee89-455a-a33c-26012078549f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860479614976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:21,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3157c16-ee89-455a-a33c-26012078549f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:45:21,017::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:45:21,017::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:45:21,017::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:45:21,017::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:45:21,018::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:45:21,018::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:45:21,018::task::993::Storage.TaskManager.Task::(_decref) Task=`f3157c16-ee89-455a-a33c-26012078549f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:45:22,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:22,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013166 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:22,543::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:22,543::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:22,543::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:22,544::__init__::232::IOProcess::(_processLogs) (5647) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:22,545::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5647) Queuing response >Thread-16::DEBUG::2014-08-11 14:45:22,545::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:45:22,564::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422754 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:22,565::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:22,566::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:22,566::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:22,566::__init__::232::IOProcess::(_processLogs) (5648) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:22,567::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5648) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:22,568::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:22,569::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:22,569::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:22,569::__init__::232::IOProcess::(_processLogs) (5649) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:22,570::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5649) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:22,571::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:22,571::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:22,571::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:22,572::__init__::232::IOProcess::(_processLogs) (5650) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:22,572::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5650) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:22,573::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:22,573::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:22,574::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:22,574::__init__::232::IOProcess::(_processLogs) (5651) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:22,574::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5651) Queuing response >Dummy-33::DEBUG::2014-08-11 14:45:24,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:24,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131713 s, 77.7 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:45:25,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`3921891e-115b-4c0e-bf81-bd493d3591f8`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:45:25,957::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:45:25,963::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422754', 'lastCheck': '3.4', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:45:25,963::task::1191::Storage.TaskManager.Task::(prepare) Task=`3921891e-115b-4c0e-bf81-bd493d3591f8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422754', 'lastCheck': '3.4', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:45:25,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`3921891e-115b-4c0e-bf81-bd493d3591f8`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:45:25,963::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:45:25,963::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:45:25,964::task::993::Storage.TaskManager.Task::(_decref) Task=`3921891e-115b-4c0e-bf81-bd493d3591f8`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:25,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:25,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:25,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:25,982::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:45:26,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:26,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132724 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:28,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:28,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132147 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:30,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:30,614::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133795 s, 76.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:30,982::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:30,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:31,032::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:31,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:31,068::task::595::Storage.TaskManager.Task::(_updateState) Task=`286d769a-9b3c-4915-a851-1b87fad77e56`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:45:31,069::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:45:31,083::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:31,084::task::1191::Storage.TaskManager.Task::(prepare) Task=`286d769a-9b3c-4915-a851-1b87fad77e56`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:45:31,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`286d769a-9b3c-4915-a851-1b87fad77e56`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:45:31,084::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:45:31,084::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:45:31,084::task::993::Storage.TaskManager.Task::(_decref) Task=`286d769a-9b3c-4915-a851-1b87fad77e56`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:45:31,095::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:45:31,095::task::595::Storage.TaskManager.Task::(_updateState) Task=`2594734b-9f44-4abf-b18f-e8923f51162b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:45:31,096::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:45:31,097::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cc78df2a-66be-4bc4-b84e-7037408898c7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:45:31,097::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:45:31,097::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:45:31,098::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cc78df2a-66be-4bc4-b84e-7037408898c7`::Granted request >Thread-34::DEBUG::2014-08-11 14:45:31,098::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2594734b-9f44-4abf-b18f-e8923f51162b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:45:31,098::task::993::Storage.TaskManager.Task::(_decref) Task=`2594734b-9f44-4abf-b18f-e8923f51162b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:45:31,119::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860461985792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:31,119::task::1191::Storage.TaskManager.Task::(prepare) Task=`2594734b-9f44-4abf-b18f-e8923f51162b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860461985792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:31,119::task::595::Storage.TaskManager.Task::(_updateState) Task=`2594734b-9f44-4abf-b18f-e8923f51162b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:45:31,119::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:45:31,120::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:45:31,120::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:45:31,120::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:45:31,121::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:45:31,121::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:45:31,121::task::993::Storage.TaskManager.Task::(_decref) Task=`2594734b-9f44-4abf-b18f-e8923f51162b`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:45:32,577::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:32,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:32,579::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:32,581::__init__::232::IOProcess::(_processLogs) (5652) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:32,583::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5652) Queuing response >Thread-16::DEBUG::2014-08-11 14:45:32,584::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:45:32,612::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00811801 s, 56.0 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:32,615::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:32,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:32,617::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:32,619::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5653) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:32,621::__init__::232::IOProcess::(_processLogs) (5653) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:32,623::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:32,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:32,626::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:32,627::__init__::232::IOProcess::(_processLogs) (5654) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:32,629::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5654) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:32,632::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:32,633::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:32,635::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:32,637::__init__::232::IOProcess::(_processLogs) (5655) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:45:32,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 14:45:32,658::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5655) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:32,660::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:32,661::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:32,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:32,663::__init__::232::IOProcess::(_processLogs) (5656) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:32,664::__init__::232::IOProcess::(_processLogs) (5656) Queuing response >Dummy-33::DEBUG::2014-08-11 14:45:32,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0422699 s, 24.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:34,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:34,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131836 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:36,073::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:36,075::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:36,163::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:45:36,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:36,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0233727 s, 43.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:39,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:39,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131387 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:41,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-97::DEBUG::2014-08-11 14:45:41,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c943a34-9dc6-437d-a696-9da9574a186f`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:45:41,087::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Dummy-33::DEBUG::2014-08-11 14:45:41,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0242523 s, 42.2 MB/s\n'; <rc> = 0 >Detector thread::DEBUG::2014-08-11 14:45:41,138::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:45085 >Thread-97::INFO::2014-08-11 14:45:41,164::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00811801', 'lastCheck': '8.5', 'valid': True}} >Detector thread::DEBUG::2014-08-11 14:45:41,165::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:45085 >Thread-97::DEBUG::2014-08-11 14:45:41,166::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c943a34-9dc6-437d-a696-9da9574a186f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00811801', 'lastCheck': '8.5', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:45:41,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c943a34-9dc6-437d-a696-9da9574a186f`::moving from state preparing -> state finished >Detector thread::DEBUG::2014-08-11 14:45:41,167::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:45085 >Thread-97::DEBUG::2014-08-11 14:45:41,169::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Detector thread::DEBUG::2014-08-11 14:45:41,171::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 45085) >Thread-97::DEBUG::2014-08-11 14:45:41,174::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:41,175::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:45:41,177::task::993::Storage.TaskManager.Task::(_decref) Task=`0c943a34-9dc6-437d-a696-9da9574a186f`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:41,189::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:41,200::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-102::DEBUG::2014-08-11 14:45:41,225::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-102::DEBUG::2014-08-11 14:45:41,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`aecd9915-7ad6-42f8-b637-5b12726b853d`::moving from state init -> state preparing >Thread-102::INFO::2014-08-11 14:45:41,228::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::DEBUG::2014-08-11 14:45:41,235::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-102::INFO::2014-08-11 14:45:41,269::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-102::DEBUG::2014-08-11 14:45:41,271::task::1191::Storage.TaskManager.Task::(prepare) Task=`aecd9915-7ad6-42f8-b637-5b12726b853d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-102::DEBUG::2014-08-11 14:45:41,272::task::595::Storage.TaskManager.Task::(_updateState) Task=`aecd9915-7ad6-42f8-b637-5b12726b853d`::moving from state preparing -> state finished >Thread-102::DEBUG::2014-08-11 14:45:41,273::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2014-08-11 14:45:41,274::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2014-08-11 14:45:41,275::task::993::Storage.TaskManager.Task::(_decref) Task=`aecd9915-7ad6-42f8-b637-5b12726b853d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:45:41,287::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:45:41,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7519cad-d75e-4c31-b030-e1d0fe153c26`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:45:41,288::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:45:41,289::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`96e381f6-e599-42b2-98be-2be1b12cb824`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:45:41,289::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:45:41,290::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:45:41,290::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`96e381f6-e599-42b2-98be-2be1b12cb824`::Granted request >Thread-34::DEBUG::2014-08-11 14:45:41,290::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f7519cad-d75e-4c31-b030-e1d0fe153c26`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:45:41,291::task::993::Storage.TaskManager.Task::(_decref) Task=`f7519cad-d75e-4c31-b030-e1d0fe153c26`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:45:41,311::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860461985792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:41,311::task::1191::Storage.TaskManager.Task::(prepare) Task=`f7519cad-d75e-4c31-b030-e1d0fe153c26`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860461985792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:41,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7519cad-d75e-4c31-b030-e1d0fe153c26`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:45:41,312::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:45:41,312::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:45:41,312::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:45:41,312::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:45:41,313::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:45:41,313::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:45:41,313::task::993::Storage.TaskManager.Task::(_decref) Task=`f7519cad-d75e-4c31-b030-e1d0fe153c26`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:45:42,663::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:42,663::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:42,664::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:42,664::__init__::232::IOProcess::(_processLogs) (5657) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:42,664::__init__::232::IOProcess::(_processLogs) (5657) Queuing response >Thread-16::DEBUG::2014-08-11 14:45:42,665::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:45:42,683::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041453 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:42,684::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:42,684::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:42,685::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:42,685::__init__::232::IOProcess::(_processLogs) (5658) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:42,686::__init__::232::IOProcess::(_processLogs) DEBUG|(5658) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:42,687::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:42,688::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:42,688::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:42,688::__init__::232::IOProcess::(_processLogs) (5659) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:42,688::__init__::232::IOProcess::(_processLogs) (5659) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:42,689::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:42,689::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:42,690::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:42,691::__init__::232::IOProcess::(_processLogs) (5660) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:42,691::__init__::232::IOProcess::(_processLogs) (5660) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:42,691::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:42,692::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:42,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:42,693::__init__::232::IOProcess::(_processLogs) (5661) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:42,693::__init__::232::IOProcess::(_processLogs) (5661) Queuing response >Dummy-33::DEBUG::2014-08-11 14:45:43,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:43,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131782 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:45,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:45,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131816 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:46,214::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:46,215::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:46,263::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:45:47,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:47,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131147 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:49,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:49,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132917 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:51,268::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:51,269::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-102::DEBUG::2014-08-11 14:45:51,328::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-102::DEBUG::2014-08-11 14:45:51,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1bc41cc-7cc3-4311-9301-921bc64a93f4`::moving from state init -> state preparing >Thread-102::INFO::2014-08-11 14:45:51,367::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:51,368::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-102::INFO::2014-08-11 14:45:51,381::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-102::DEBUG::2014-08-11 14:45:51,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`a1bc41cc-7cc3-4311-9301-921bc64a93f4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-102::DEBUG::2014-08-11 14:45:51,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1bc41cc-7cc3-4311-9301-921bc64a93f4`::moving from state preparing -> state finished >Thread-102::DEBUG::2014-08-11 14:45:51,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2014-08-11 14:45:51,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2014-08-11 14:45:51,382::task::993::Storage.TaskManager.Task::(_decref) Task=`a1bc41cc-7cc3-4311-9301-921bc64a93f4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:45:51,393::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:45:51,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`bce8d131-9dcc-4f12-abe9-9ce4dd1ef7f6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:45:51,394::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:45:51,395::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f6d38001-5bdd-46c7-93f7-e5e8178da38d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:45:51,395::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:45:51,396::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:45:51,396::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f6d38001-5bdd-46c7-93f7-e5e8178da38d`::Granted request >Thread-34::DEBUG::2014-08-11 14:45:51,397::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bce8d131-9dcc-4f12-abe9-9ce4dd1ef7f6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:45:51,397::task::993::Storage.TaskManager.Task::(_decref) Task=`bce8d131-9dcc-4f12-abe9-9ce4dd1ef7f6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:45:51,417::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860461985792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:51,417::task::1191::Storage.TaskManager.Task::(prepare) Task=`bce8d131-9dcc-4f12-abe9-9ce4dd1ef7f6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860461985792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:45:51,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`bce8d131-9dcc-4f12-abe9-9ce4dd1ef7f6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:45:51,418::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:45:51,418::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:45:51,418::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:45:51,418::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:45:51,419::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:45:51,419::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:45:51,419::task::993::Storage.TaskManager.Task::(_decref) Task=`bce8d131-9dcc-4f12-abe9-9ce4dd1ef7f6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:45:51,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:51,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132956 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:52,694::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:52,694::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:52,695::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:45:52,696::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:45:52,697::__init__::232::IOProcess::(_processLogs) (5662) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:52,714::__init__::232::IOProcess::(_processLogs) (5662) Queuing response >Thread-16::DEBUG::2014-08-11 14:45:52,715::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432293 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:45:52,716::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:52,717::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:52,717::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:52,717::__init__::232::IOProcess::(_processLogs) (5663) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:45:52,718::__init__::232::IOProcess::(_processLogs) DEBUG|(5663) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:52,720::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:52,720::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:52,721::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:52,721::__init__::232::IOProcess::(_processLogs) (5664) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:52,721::__init__::232::IOProcess::(_processLogs) (5664) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:52,722::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:52,722::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:52,723::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:52,723::__init__::232::IOProcess::(_processLogs) (5665) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:52,723::__init__::232::IOProcess::(_processLogs) (5665) Queuing response >Thread-14::DEBUG::2014-08-11 14:45:52,724::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:45:52,724::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:45:52,726::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:45:52,726::__init__::232::IOProcess::(_processLogs) (5666) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:45:52,727::__init__::232::IOProcess::(_processLogs) (5666) Queuing response >Dummy-33::DEBUG::2014-08-11 14:45:53,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:53,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131724 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:55,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:55,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013183 s, 77.7 MB/s\n'; <rc> = 0 >Thread-102::DEBUG::2014-08-11 14:45:56,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`12dbec13-9798-497e-a5b2-d110be565b2a`::moving from state init -> state preparing >Thread-102::INFO::2014-08-11 14:45:56,355::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-102::INFO::2014-08-11 14:45:56,363::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432293', 'lastCheck': '3.6', 'valid': True}} >Thread-102::DEBUG::2014-08-11 14:45:56,363::task::1191::Storage.TaskManager.Task::(prepare) Task=`12dbec13-9798-497e-a5b2-d110be565b2a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432293', 'lastCheck': '3.6', 'valid': True}} >Thread-102::DEBUG::2014-08-11 14:45:56,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`12dbec13-9798-497e-a5b2-d110be565b2a`::moving from state preparing -> state finished >Thread-102::DEBUG::2014-08-11 14:45:56,363::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2014-08-11 14:45:56,363::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2014-08-11 14:45:56,364::task::993::Storage.TaskManager.Task::(_decref) Task=`12dbec13-9798-497e-a5b2-d110be565b2a`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:56,375::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:56,376::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:45:56,377::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-102::DEBUG::2014-08-11 14:45:56,384::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:45:57,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:57,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131629 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:45:59,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:45:59,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131984 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:01,385::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:01,393::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:01,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-102::DEBUG::2014-08-11 14:46:01,471::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-102::DEBUG::2014-08-11 14:46:01,472::task::595::Storage.TaskManager.Task::(_updateState) Task=`53d8435d-319a-4ec9-bbf4-66219bf1bc20`::moving from state init -> state preparing >Thread-102::INFO::2014-08-11 14:46:01,472::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-102::INFO::2014-08-11 14:46:01,487::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-102::DEBUG::2014-08-11 14:46:01,487::task::1191::Storage.TaskManager.Task::(prepare) Task=`53d8435d-319a-4ec9-bbf4-66219bf1bc20`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-102::DEBUG::2014-08-11 14:46:01,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`53d8435d-319a-4ec9-bbf4-66219bf1bc20`::moving from state preparing -> state finished >Thread-102::DEBUG::2014-08-11 14:46:01,487::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2014-08-11 14:46:01,488::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2014-08-11 14:46:01,488::task::993::Storage.TaskManager.Task::(_decref) Task=`53d8435d-319a-4ec9-bbf4-66219bf1bc20`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:46:01,499::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:46:01,500::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d7003a2-9e35-46f9-81e2-f14088f2b010`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:46:01,500::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:46:01,501::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ecfff4d1-ca2b-4b7e-82e9-d2809d217a9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:46:01,501::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:46:01,501::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:46:01,502::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ecfff4d1-ca2b-4b7e-82e9-d2809d217a9c`::Granted request >Thread-34::DEBUG::2014-08-11 14:46:01,502::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2d7003a2-9e35-46f9-81e2-f14088f2b010`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:46:01,502::task::993::Storage.TaskManager.Task::(_decref) Task=`2d7003a2-9e35-46f9-81e2-f14088f2b010`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:46:01,523::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860446191616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:01,523::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d7003a2-9e35-46f9-81e2-f14088f2b010`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860446191616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:01,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d7003a2-9e35-46f9-81e2-f14088f2b010`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:46:01,524::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:46:01,524::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:46:01,524::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:46:01,525::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:46:01,525::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:46:01,525::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:46:01,526::task::993::Storage.TaskManager.Task::(_decref) Task=`2d7003a2-9e35-46f9-81e2-f14088f2b010`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:46:01,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:01,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133211 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:02,728::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:02,730::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:02,731::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:02,732::__init__::232::IOProcess::(_processLogs) (5667) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:02,734::__init__::232::IOProcess::(_processLogs) DEBUG|(5667) Queuing response >Thread-16::DEBUG::2014-08-11 14:46:02,736::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:46:02,764::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00706027 s, 64.4 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:02,767::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:02,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:02,770::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:02,771::__init__::232::IOProcess::(_processLogs) DEBUG|(5668) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:02,773::__init__::232::IOProcess::(_processLogs) (5668) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:02,775::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:02,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:02,778::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:02,779::__init__::232::IOProcess::(_processLogs) (5669) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:02,781::__init__::232::IOProcess::(_processLogs) (5669) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:02,782::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:02,784::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:02,786::__init__::232::IOProcess::(_processLogs) (5670) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:02,787::__init__::232::IOProcess::(_processLogs) (5670) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:02,789::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:02,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:02,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:02,793::__init__::232::IOProcess::(_processLogs) (5671) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:02,794::__init__::232::IOProcess::(_processLogs) (5671) Queuing response >Dummy-33::DEBUG::2014-08-11 14:46:03,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:03,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132749 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:05,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:05,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132165 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:06,473::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:06,474::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:06,565::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:46:07,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:07,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132019 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:09,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:10,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01321 s, 77.5 MB/s\n'; <rc> = 0 >Thread-102::DEBUG::2014-08-11 14:46:11,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8a54ba5-c284-4562-b4ef-9d31f1ee6f87`::moving from state init -> state preparing >Thread-102::INFO::2014-08-11 14:46:11,475::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::DEBUG::2014-08-11 14:46:11,540::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-102::INFO::2014-08-11 14:46:11,566::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00706027', 'lastCheck': '8.8', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:46:11,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`1269565f-3f74-4bec-b408-5c4062616dad`::moving from state init -> state preparing >Thread-102::DEBUG::2014-08-11 14:46:11,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8a54ba5-c284-4562-b4ef-9d31f1ee6f87`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00706027', 'lastCheck': '8.8', 'valid': True}} >Thread-97::INFO::2014-08-11 14:46:11,568::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-102::DEBUG::2014-08-11 14:46:11,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8a54ba5-c284-4562-b4ef-9d31f1ee6f87`::moving from state preparing -> state finished >Thread-102::DEBUG::2014-08-11 14:46:11,570::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2014-08-11 14:46:11,570::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2014-08-11 14:46:11,570::task::993::Storage.TaskManager.Task::(_decref) Task=`e8a54ba5-c284-4562-b4ef-9d31f1ee6f87`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:11,579::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:11,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:11,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:46:11,585::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:11,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`1269565f-3f74-4bec-b408-5c4062616dad`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:11,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`1269565f-3f74-4bec-b408-5c4062616dad`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:46:11,585::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:11,586::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:46:11,586::task::993::Storage.TaskManager.Task::(_decref) Task=`1269565f-3f74-4bec-b408-5c4062616dad`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:46:11,596::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:46:11,596::task::595::Storage.TaskManager.Task::(_updateState) Task=`5aa6f6b1-571f-49c4-9700-9e059545b8e7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:46:11,597::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:46:11,597::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`960e677a-7e76-4207-be87-d38528c1c26e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:46:11,598::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:46:11,598::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:46:11,598::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`960e677a-7e76-4207-be87-d38528c1c26e`::Granted request >Thread-34::DEBUG::2014-08-11 14:46:11,599::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5aa6f6b1-571f-49c4-9700-9e059545b8e7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:46:11,599::task::993::Storage.TaskManager.Task::(_decref) Task=`5aa6f6b1-571f-49c4-9700-9e059545b8e7`::ref 1 aborting False >Thread-97::DEBUG::2014-08-11 14:46:11,605::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:46:11,637::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860401954816', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:11,638::task::1191::Storage.TaskManager.Task::(prepare) Task=`5aa6f6b1-571f-49c4-9700-9e059545b8e7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860401954816', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:11,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`5aa6f6b1-571f-49c4-9700-9e059545b8e7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:46:11,638::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:46:11,639::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:46:11,639::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:46:11,639::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:46:11,640::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:46:11,640::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:46:11,640::task::993::Storage.TaskManager.Task::(_decref) Task=`5aa6f6b1-571f-49c4-9700-9e059545b8e7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:46:12,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:12,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132301 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:12,791::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:12,792::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:12,792::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:12,792::__init__::232::IOProcess::(_processLogs) (5672) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:12,793::__init__::232::IOProcess::(_processLogs) DEBUG|(5672) Queuing response >Thread-16::DEBUG::2014-08-11 14:46:12,794::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:46:12,813::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416567 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:12,814::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:12,814::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:12,814::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:12,815::__init__::232::IOProcess::(_processLogs) (5673) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:12,815::__init__::232::IOProcess::(_processLogs) DEBUG|(5673) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:12,817::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:12,817::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:12,817::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:12,817::__init__::232::IOProcess::(_processLogs) (5674) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:12,818::__init__::232::IOProcess::(_processLogs) DEBUG|(5674) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:12,819::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:12,819::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:12,820::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:12,821::__init__::232::IOProcess::(_processLogs) (5675) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:12,821::__init__::232::IOProcess::(_processLogs) (5675) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:12,821::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:12,822::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:12,822::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:12,822::__init__::232::IOProcess::(_processLogs) (5676) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:12,823::__init__::232::IOProcess::(_processLogs) DEBUG|(5676) Queuing response >Thread-55::DEBUG::2014-08-11 14:46:12,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b1792f8-824c-4442-988a-42a7e46fb9c9`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:46:12,853::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:46:12,854::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:12,854::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:12,855::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:12,855::__init__::232::IOProcess::(_processLogs) (5677) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:46:12,856::__init__::232::IOProcess::(_processLogs) DEBUG|(5677) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:12,857::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:12,857::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:12,857::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:12,857::__init__::232::IOProcess::(_processLogs) (5678) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:46:12,858::__init__::232::IOProcess::(_processLogs) DEBUG|(5678) Queuing response >Thread-55::INFO::2014-08-11 14:46:12,858::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:46:12,859::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b1792f8-824c-4442-988a-42a7e46fb9c9`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:46:12,860::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b1792f8-824c-4442-988a-42a7e46fb9c9`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:46:12,860::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:46:12,860::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:46:12,860::task::993::Storage.TaskManager.Task::(_decref) Task=`0b1792f8-824c-4442-988a-42a7e46fb9c9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:46:14,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:14,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132462 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:16,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:16,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133281 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:16,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:16,592::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:16,665::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:46:18,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:18,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131654 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:20,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:20,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133089 s, 76.9 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:46:21,651::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:46:21,665::task::595::Storage.TaskManager.Task::(_updateState) Task=`bcb7334b-5807-4300-9549-21ddffca00ad`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:46:21,665::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:21,672::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:21,674::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:21,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:46:21,678::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:21,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`bcb7334b-5807-4300-9549-21ddffca00ad`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:21,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`bcb7334b-5807-4300-9549-21ddffca00ad`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:46:21,679::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:21,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:46:21,679::task::993::Storage.TaskManager.Task::(_decref) Task=`bcb7334b-5807-4300-9549-21ddffca00ad`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:46:21,689::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:46:21,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ba75657-e0e8-489d-bebb-f1cc37047bf5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:46:21,690::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:46:21,691::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8c14730a-79d1-408a-a200-68ebcb192d20`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:46:21,691::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:46:21,691::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:46:21,692::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8c14730a-79d1-408a-a200-68ebcb192d20`::Granted request >Thread-34::DEBUG::2014-08-11 14:46:21,692::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2ba75657-e0e8-489d-bebb-f1cc37047bf5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:46:21,692::task::993::Storage.TaskManager.Task::(_decref) Task=`2ba75657-e0e8-489d-bebb-f1cc37047bf5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:46:21,712::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860393566208', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:21,712::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ba75657-e0e8-489d-bebb-f1cc37047bf5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860393566208', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:21,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ba75657-e0e8-489d-bebb-f1cc37047bf5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:46:21,713::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:46:21,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:46:21,713::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:46:21,714::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:46:21,714::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:46:21,714::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:46:21,714::task::993::Storage.TaskManager.Task::(_decref) Task=`2ba75657-e0e8-489d-bebb-f1cc37047bf5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:46:22,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:22,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131857 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:22,824::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:22,825::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:22,825::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:22,825::__init__::232::IOProcess::(_processLogs) (5679) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:22,826::__init__::232::IOProcess::(_processLogs) DEBUG|(5679) Queuing response >Thread-16::DEBUG::2014-08-11 14:46:22,827::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:46:22,846::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420455 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:22,847::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:22,848::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:22,848::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:22,848::__init__::232::IOProcess::(_processLogs) (5680) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:22,849::__init__::232::IOProcess::(_processLogs) DEBUG|(5680) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:22,851::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:22,851::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:22,852::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:22,852::__init__::232::IOProcess::(_processLogs) (5681) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:22,852::__init__::232::IOProcess::(_processLogs) DEBUG|(5681) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:22,853::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:22,853::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:22,854::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:22,854::__init__::232::IOProcess::(_processLogs) (5682) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:22,855::__init__::232::IOProcess::(_processLogs) DEBUG|(5682) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:22,856::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:22,856::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:22,856::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:22,856::__init__::232::IOProcess::(_processLogs) (5683) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:22,857::__init__::232::IOProcess::(_processLogs) DEBUG|(5683) Queuing response >Dummy-33::DEBUG::2014-08-11 14:46:24,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:24,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0196741 s, 52.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:26,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:26,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132406 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:26,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:26,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:46:26,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`53650b88-76cd-45d3-848b-eb3674af4c4a`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:46:26,766::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:46:26,768::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420455', 'lastCheck': '3.9', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:26,770::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:46:26,770::task::1191::Storage.TaskManager.Task::(prepare) Task=`53650b88-76cd-45d3-848b-eb3674af4c4a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420455', 'lastCheck': '3.9', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:46:26,772::task::595::Storage.TaskManager.Task::(_updateState) Task=`53650b88-76cd-45d3-848b-eb3674af4c4a`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:46:26,773::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:26,774::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:46:26,774::task::993::Storage.TaskManager.Task::(_decref) Task=`53650b88-76cd-45d3-848b-eb3674af4c4a`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:46:26,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:46:28,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:28,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131969 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:30,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:30,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133729 s, 76.6 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:46:31,726::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-97::DEBUG::2014-08-11 14:46:31,766::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdcc1f55-8f96-4a0f-b4f0-5af9d651e732`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:46:31,766::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:31,780::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:31,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:31,784::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:46:31,784::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:31,785::task::1191::Storage.TaskManager.Task::(prepare) Task=`cdcc1f55-8f96-4a0f-b4f0-5af9d651e732`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:31,787::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdcc1f55-8f96-4a0f-b4f0-5af9d651e732`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:46:31,787::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:31,787::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:46:31,788::task::993::Storage.TaskManager.Task::(_decref) Task=`cdcc1f55-8f96-4a0f-b4f0-5af9d651e732`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:46:31,799::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-34::DEBUG::2014-08-11 14:46:31,799::task::595::Storage.TaskManager.Task::(_updateState) Task=`f89656bb-6314-40ee-a765-19635fb82a14`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:46:31,800::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:46:31,800::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5eb08adc-78ab-4a53-8ca4-3748c7752d0d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:46:31,801::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:46:31,801::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:46:31,801::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5eb08adc-78ab-4a53-8ca4-3748c7752d0d`::Granted request >Thread-34::DEBUG::2014-08-11 14:46:31,803::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f89656bb-6314-40ee-a765-19635fb82a14`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:46:31,803::task::993::Storage.TaskManager.Task::(_decref) Task=`f89656bb-6314-40ee-a765-19635fb82a14`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:46:31,823::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860393566208', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:31,824::task::1191::Storage.TaskManager.Task::(prepare) Task=`f89656bb-6314-40ee-a765-19635fb82a14`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860393566208', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:31,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`f89656bb-6314-40ee-a765-19635fb82a14`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:46:31,824::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:46:31,825::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:46:31,825::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:46:31,825::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:46:31,826::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:46:31,826::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:46:31,826::task::993::Storage.TaskManager.Task::(_decref) Task=`f89656bb-6314-40ee-a765-19635fb82a14`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:46:32,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:32,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.029818 s, 34.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:32,860::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:32,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:32,863::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:32,864::__init__::232::IOProcess::(_processLogs) (5684) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:32,867::__init__::232::IOProcess::(_processLogs) DEBUG|(5684) Queuing response >Thread-16::DEBUG::2014-08-11 14:46:32,869::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:46:32,891::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419848 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:32,895::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:32,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:32,897::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:32,898::__init__::232::IOProcess::(_processLogs) (5685) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:32,900::__init__::232::IOProcess::(_processLogs) (5685) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:32,905::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:32,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:32,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:32,910::__init__::232::IOProcess::(_processLogs) (5686) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:32,911::__init__::232::IOProcess::(_processLogs) (5686) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:32,914::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:32,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:32,917::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:32,918::__init__::232::IOProcess::(_processLogs) (5687) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:32,919::__init__::232::IOProcess::(_processLogs) (5687) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:32,922::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:32,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:32,927::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:32,928::__init__::232::IOProcess::(_processLogs) (5688) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:32,930::__init__::232::IOProcess::(_processLogs) (5688) Queuing response >Dummy-33::DEBUG::2014-08-11 14:46:34,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:34,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0159529 s, 64.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:36,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:36,866::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:36,867::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:46:37,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:37,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132371 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:39,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:39,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131803 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:41,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:41,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134274 s, 76.3 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:46:41,840::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:46:41,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a8a65fb-e469-46d9-93d3-e4ec406c3569`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:46:41,865::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:41,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:41,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:41,876::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:46:41,878::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:41,879::task::1191::Storage.TaskManager.Task::(prepare) Task=`9a8a65fb-e469-46d9-93d3-e4ec406c3569`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:41,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a8a65fb-e469-46d9-93d3-e4ec406c3569`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:46:41,879::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:41,879::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:46:41,879::task::993::Storage.TaskManager.Task::(_decref) Task=`9a8a65fb-e469-46d9-93d3-e4ec406c3569`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:46:41,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7b4209e-7ca2-4474-9845-79cb43be19fe`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:46:41,893::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-34::DEBUG::2014-08-11 14:46:41,894::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::INFO::2014-08-11 14:46:41,895::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419848', 'lastCheck': '9.0', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:46:41,896::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7b4209e-7ca2-4474-9845-79cb43be19fe`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419848', 'lastCheck': '9.0', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:46:41,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7b4209e-7ca2-4474-9845-79cb43be19fe`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:46:41,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`f68a8c1a-d3ea-461e-88b0-43b0583fd287`::moving from state init -> state preparing >Thread-97::DEBUG::2014-08-11 14:46:41,898::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:41,899::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::INFO::2014-08-11 14:46:41,899::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::DEBUG::2014-08-11 14:46:41,900::task::993::Storage.TaskManager.Task::(_decref) Task=`e7b4209e-7ca2-4474-9845-79cb43be19fe`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:46:41,904::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`840e0acb-7fb6-4936-9d07-baf998107768`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:46:41,906::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:46:41,908::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:46:41,910::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`840e0acb-7fb6-4936-9d07-baf998107768`::Granted request >Thread-34::DEBUG::2014-08-11 14:46:41,914::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f68a8c1a-d3ea-461e-88b0-43b0583fd287`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:46:41,916::task::993::Storage.TaskManager.Task::(_decref) Task=`f68a8c1a-d3ea-461e-88b0-43b0583fd287`::ref 1 aborting False >Thread-97::DEBUG::2014-08-11 14:46:41,937::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 14:46:41,940::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860387864576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:41,942::task::1191::Storage.TaskManager.Task::(prepare) Task=`f68a8c1a-d3ea-461e-88b0-43b0583fd287`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860387864576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:41,943::task::595::Storage.TaskManager.Task::(_updateState) Task=`f68a8c1a-d3ea-461e-88b0-43b0583fd287`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:46:41,944::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:46:41,944::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:46:41,945::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:46:41,945::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:46:41,945::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:46:41,945::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:46:41,946::task::993::Storage.TaskManager.Task::(_decref) Task=`f68a8c1a-d3ea-461e-88b0-43b0583fd287`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:46:42,925::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:42,926::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:42,926::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:42,926::__init__::232::IOProcess::(_processLogs) (5689) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:42,927::__init__::232::IOProcess::(_processLogs) DEBUG|(5689) Queuing response >Thread-16::DEBUG::2014-08-11 14:46:42,928::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:46:42,947::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000968933 s, 470 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:42,949::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:42,950::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:42,951::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:42,951::__init__::232::IOProcess::(_processLogs) (5690) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:42,952::__init__::232::IOProcess::(_processLogs) (5690) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:42,952::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:42,953::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:42,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:42,954::__init__::232::IOProcess::(_processLogs) (5691) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:42,954::__init__::232::IOProcess::(_processLogs) (5691) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:42,955::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:42,955::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:42,956::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:42,957::__init__::232::IOProcess::(_processLogs) (5692) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:42,957::__init__::232::IOProcess::(_processLogs) (5692) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:42,958::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:42,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:42,959::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:42,959::__init__::232::IOProcess::(_processLogs) (5693) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:42,960::__init__::232::IOProcess::(_processLogs) (5693) Queuing response >Dummy-33::DEBUG::2014-08-11 14:46:43,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:43,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131548 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:45,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:45,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132108 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:46,883::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:46,884::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:46,966::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:46:47,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:47,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136687 s, 74.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:49,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:49,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132581 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:51,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:51,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132507 s, 77.3 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:46:51,960::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:46:51,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`e709c5b2-a43a-49a9-97fb-b6f5950ab767`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:46:51,966::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:51,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:51,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:51,976::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::INFO::2014-08-11 14:46:51,979::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:51,980::task::1191::Storage.TaskManager.Task::(prepare) Task=`e709c5b2-a43a-49a9-97fb-b6f5950ab767`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:46:51,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`e709c5b2-a43a-49a9-97fb-b6f5950ab767`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:46:51,980::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:51,980::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:46:51,980::task::993::Storage.TaskManager.Task::(_decref) Task=`e709c5b2-a43a-49a9-97fb-b6f5950ab767`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:46:51,991::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:46:51,991::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8b53b65-7cf3-48db-98d6-bfb09f4e9a0c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:46:51,992::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:46:51,992::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7ac06a3f-0632-4230-9f14-29c291879fc1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:46:51,993::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:46:51,993::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:46:51,993::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7ac06a3f-0632-4230-9f14-29c291879fc1`::Granted request >Thread-34::DEBUG::2014-08-11 14:46:51,994::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b8b53b65-7cf3-48db-98d6-bfb09f4e9a0c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:46:51,994::task::993::Storage.TaskManager.Task::(_decref) Task=`b8b53b65-7cf3-48db-98d6-bfb09f4e9a0c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:46:52,022::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860378820608', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:52,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8b53b65-7cf3-48db-98d6-bfb09f4e9a0c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860378820608', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:46:52,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8b53b65-7cf3-48db-98d6-bfb09f4e9a0c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:46:52,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:46:52,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:46:52,024::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:46:52,024::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:46:52,024::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:46:52,024::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:46:52,025::task::993::Storage.TaskManager.Task::(_decref) Task=`b8b53b65-7cf3-48db-98d6-bfb09f4e9a0c`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:46:52,961::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:52,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:52,962::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:52,962::__init__::232::IOProcess::(_processLogs) (5694) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:52,962::__init__::232::IOProcess::(_processLogs) (5694) Queuing response >Thread-16::DEBUG::2014-08-11 14:46:52,963::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:46:52,983::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000435282 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:46:52,984::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:52,984::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:52,984::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:52,985::__init__::232::IOProcess::(_processLogs) (5695) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:46:52,985::__init__::232::IOProcess::(_processLogs) DEBUG|(5695) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:52,987::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:52,987::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:52,988::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:52,988::__init__::232::IOProcess::(_processLogs) (5696) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:52,988::__init__::232::IOProcess::(_processLogs) (5696) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:52,989::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:52,989::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:52,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:52,990::__init__::232::IOProcess::(_processLogs) (5697) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:52,991::__init__::232::IOProcess::(_processLogs) DEBUG|(5697) Queuing response >Thread-14::DEBUG::2014-08-11 14:46:52,991::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:46:52,992::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:46:52,993::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:46:52,993::__init__::232::IOProcess::(_processLogs) (5698) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:46:52,993::__init__::232::IOProcess::(_processLogs) (5698) Queuing response >Dummy-33::DEBUG::2014-08-11 14:46:53,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:53,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131716 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:55,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:55,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132125 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:56,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:56,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:46:57,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`9daced16-b289-474d-86ae-fa6480cdf789`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:46:57,067::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:46:57,070::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435282', 'lastCheck': '4.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:46:57,072::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:46:57,073::task::1191::Storage.TaskManager.Task::(prepare) Task=`9daced16-b289-474d-86ae-fa6480cdf789`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435282', 'lastCheck': '4.1', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:46:57,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`9daced16-b289-474d-86ae-fa6480cdf789`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:46:57,076::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:46:57,077::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:46:57,077::task::993::Storage.TaskManager.Task::(_decref) Task=`9daced16-b289-474d-86ae-fa6480cdf789`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:46:57,097::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:46:57,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:57,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131787 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:46:59,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:46:59,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013234 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:01,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:01,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133236 s, 76.9 MB/s\n'; <rc> = 0 >Thread-97::DEBUG::2014-08-11 14:47:02,039::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-97::DEBUG::2014-08-11 14:47:02,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf753d8a-e2ad-40d6-afab-19242530be84`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:47:02,066::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-97::INFO::2014-08-11 14:47:02,082::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-97::DEBUG::2014-08-11 14:47:02,085::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf753d8a-e2ad-40d6-afab-19242530be84`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:02,087::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:47:02,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf753d8a-e2ad-40d6-afab-19242530be84`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:47:02,090::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:47:02,093::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:47:02,096::task::993::Storage.TaskManager.Task::(_decref) Task=`cf753d8a-e2ad-40d6-afab-19242530be84`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:02,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:02,105::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 14:47:02,113::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:47:02,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4574970-c39d-45b8-ab65-bdf8c075eb05`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:47:02,118::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:47:02,122::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`403cfeff-973c-43ea-9439-a48de93f36a3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:47:02,123::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:47:02,125::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:47:02,126::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`403cfeff-973c-43ea-9439-a48de93f36a3`::Granted request >Thread-34::DEBUG::2014-08-11 14:47:02,129::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e4574970-c39d-45b8-ab65-bdf8c075eb05`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:47:02,130::task::993::Storage.TaskManager.Task::(_decref) Task=`e4574970-c39d-45b8-ab65-bdf8c075eb05`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:47:02,190::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860373839872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:02,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`e4574970-c39d-45b8-ab65-bdf8c075eb05`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860373839872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:02,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4574970-c39d-45b8-ab65-bdf8c075eb05`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:47:02,194::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:47:02,196::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:47:02,199::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:47:02,200::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:47:02,201::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:47:02,203::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:47:02,205::task::993::Storage.TaskManager.Task::(_decref) Task=`e4574970-c39d-45b8-ab65-bdf8c075eb05`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:47:02,993::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:02,994::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:02,994::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:02,994::__init__::232::IOProcess::(_processLogs) (5699) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:02,995::__init__::232::IOProcess::(_processLogs) DEBUG|(5699) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:02,996::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:47:03,015::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420909 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:47:03,016::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:03,017::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:03,017::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:03,017::__init__::232::IOProcess::(_processLogs) (5700) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:03,018::__init__::232::IOProcess::(_processLogs) DEBUG|(5700) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:03,019::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:03,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:03,020::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:03,021::__init__::232::IOProcess::(_processLogs) (5701) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:03,021::__init__::232::IOProcess::(_processLogs) (5701) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:03,022::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:03,022::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:03,023::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:03,023::__init__::232::IOProcess::(_processLogs) (5702) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:03,023::__init__::232::IOProcess::(_processLogs) (5702) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:03,024::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:03,024::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:03,025::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:03,026::__init__::232::IOProcess::(_processLogs) (5703) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:03,026::__init__::232::IOProcess::(_processLogs) (5703) Queuing response >Dummy-33::DEBUG::2014-08-11 14:47:03,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:03,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132955 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:05,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:05,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131763 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:07,118::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:07,119::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:07,166::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:07,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:07,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131876 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:09,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:09,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0142389 s, 71.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:11,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:11,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131398 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:12,174::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:12,175::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:47:12,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`391b57a3-41b9-4aaf-8b59-98a1ea9363dc`::moving from state init -> state preparing >Thread-97::INFO::2014-08-11 14:47:12,267::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-97::INFO::2014-08-11 14:47:12,269::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420909', 'lastCheck': '9.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:12,270::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-97::DEBUG::2014-08-11 14:47:12,271::task::1191::Storage.TaskManager.Task::(prepare) Task=`391b57a3-41b9-4aaf-8b59-98a1ea9363dc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420909', 'lastCheck': '9.2', 'valid': True}} >Thread-97::DEBUG::2014-08-11 14:47:12,272::task::595::Storage.TaskManager.Task::(_updateState) Task=`391b57a3-41b9-4aaf-8b59-98a1ea9363dc`::moving from state preparing -> state finished >Thread-97::DEBUG::2014-08-11 14:47:12,273::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-97::DEBUG::2014-08-11 14:47:12,274::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-97::DEBUG::2014-08-11 14:47:12,274::task::993::Storage.TaskManager.Task::(_decref) Task=`391b57a3-41b9-4aaf-8b59-98a1ea9363dc`::ref 0 aborting False >Thread-97::DEBUG::2014-08-11 14:47:12,293::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 14:47:12,953::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:35517 >Detector thread::DEBUG::2014-08-11 14:47:12,954::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:35517 >Detector thread::DEBUG::2014-08-11 14:47:12,954::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:35517 >Detector thread::DEBUG::2014-08-11 14:47:12,955::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 35517) >Thread-103::DEBUG::2014-08-11 14:47:12,957::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:47:12,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e8b3406-bfa0-4852-8b5f-3af7f92206e2`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:47:12,958::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:47:12,970::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:12,971::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e8b3406-bfa0-4852-8b5f-3af7f92206e2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:12,971::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e8b3406-bfa0-4852-8b5f-3af7f92206e2`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:12,971::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:12,971::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:47:12,972::task::993::Storage.TaskManager.Task::(_decref) Task=`8e8b3406-bfa0-4852-8b5f-3af7f92206e2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:47:12,983::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:47:12,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`20385368-85f8-4eb1-b5dc-156a04523c22`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:47:12,984::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:47:12,984::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4dab3023-b356-46eb-9092-98939030bc81`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:47:12,985::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:47:12,985::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:47:12,985::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4dab3023-b356-46eb-9092-98939030bc81`::Granted request >Thread-34::DEBUG::2014-08-11 14:47:12,986::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`20385368-85f8-4eb1-b5dc-156a04523c22`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:47:12,986::task::993::Storage.TaskManager.Task::(_decref) Task=`20385368-85f8-4eb1-b5dc-156a04523c22`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:47:13,006::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860365844480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:13,006::task::1191::Storage.TaskManager.Task::(prepare) Task=`20385368-85f8-4eb1-b5dc-156a04523c22`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860365844480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:13,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`20385368-85f8-4eb1-b5dc-156a04523c22`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:47:13,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:47:13,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:47:13,007::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:47:13,007::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:47:13,008::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:47:13,008::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:47:13,008::task::993::Storage.TaskManager.Task::(_decref) Task=`20385368-85f8-4eb1-b5dc-156a04523c22`::ref 0 aborting False >Thread-55::DEBUG::2014-08-11 14:47:13,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0b5ec66-a98c-482a-b95c-9db6e4db79c2`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:47:13,017::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:47:13,018::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:13,018::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:13,018::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:13,018::__init__::232::IOProcess::(_processLogs) (5704) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:47:13,019::__init__::232::IOProcess::(_processLogs) DEBUG|(5704) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:13,020::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:13,021::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:47:13,021::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:47:13,022::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:47:13,023::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0b5ec66-a98c-482a-b95c-9db6e4db79c2`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:47:13,023::__init__::232::IOProcess::(_processLogs) (5705) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:47:13,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0b5ec66-a98c-482a-b95c-9db6e4db79c2`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:47:13,024::__init__::232::IOProcess::(_processLogs) (5705) Queuing response >Thread-55::DEBUG::2014-08-11 14:47:13,025::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:47:13,026::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:47:13,027::task::993::Storage.TaskManager.Task::(_decref) Task=`a0b5ec66-a98c-482a-b95c-9db6e4db79c2`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:47:13,027::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:13,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:13,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:13,029::__init__::232::IOProcess::(_processLogs) (5706) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:13,029::__init__::232::IOProcess::(_processLogs) DEBUG|(5706) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:13,030::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:47:13,048::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000784225 s, 580 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:47:13,050::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:13,051::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:13,051::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:13,051::__init__::232::IOProcess::(_processLogs) (5707) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:13,052::__init__::232::IOProcess::(_processLogs) DEBUG|(5707) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:13,053::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:13,053::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:13,053::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:13,053::__init__::232::IOProcess::(_processLogs) (5708) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:13,054::__init__::232::IOProcess::(_processLogs) DEBUG|(5708) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:13,055::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:13,055::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:13,055::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:13,056::__init__::232::IOProcess::(_processLogs) (5709) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:13,056::__init__::232::IOProcess::(_processLogs) (5709) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:13,057::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:13,057::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:13,058::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:13,058::__init__::232::IOProcess::(_processLogs) (5710) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:13,058::__init__::232::IOProcess::(_processLogs) DEBUG|(5710) Queuing response >Dummy-33::DEBUG::2014-08-11 14:47:13,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:14,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131518 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:16,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:16,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132788 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:17,276::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:17,277::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:17,366::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:18,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:18,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132423 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:20,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:20,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244443 s, 41.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:22,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:22,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131927 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:22,373::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:22,375::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:22,376::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:47:23,020::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:47:23,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`578b4f1d-4df5-456c-b93e-9c54d29c3609`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:47:23,021::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:47:23,034::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:23,034::task::1191::Storage.TaskManager.Task::(prepare) Task=`578b4f1d-4df5-456c-b93e-9c54d29c3609`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:23,035::task::595::Storage.TaskManager.Task::(_updateState) Task=`578b4f1d-4df5-456c-b93e-9c54d29c3609`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:23,035::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:23,035::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:47:23,035::task::993::Storage.TaskManager.Task::(_decref) Task=`578b4f1d-4df5-456c-b93e-9c54d29c3609`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:47:23,045::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:47:23,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d874ddc-4f37-495c-9fb4-3da9f2785b41`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:47:23,046::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:47:23,047::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`613eb7aa-dba2-4a16-9934-b7f139872617`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:47:23,047::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:47:23,048::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:47:23,048::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`613eb7aa-dba2-4a16-9934-b7f139872617`::Granted request >Thread-34::DEBUG::2014-08-11 14:47:23,048::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3d874ddc-4f37-495c-9fb4-3da9f2785b41`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:47:23,049::task::993::Storage.TaskManager.Task::(_decref) Task=`3d874ddc-4f37-495c-9fb4-3da9f2785b41`::ref 1 aborting False >Thread-14::DEBUG::2014-08-11 14:47:23,060::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:23,061::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:23,062::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:23,062::__init__::232::IOProcess::(_processLogs) (5711) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:23,068::__init__::232::IOProcess::(_processLogs) DEBUG|(5711) Queuing response >Thread-34::INFO::2014-08-11 14:47:23,069::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860361977856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-16::DEBUG::2014-08-11 14:47:23,071::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-34::DEBUG::2014-08-11 14:47:23,071::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d874ddc-4f37-495c-9fb4-3da9f2785b41`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860361977856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:23,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d874ddc-4f37-495c-9fb4-3da9f2785b41`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:47:23,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:47:23,090::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:47:23,090::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:47:23,091::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:47:23,092::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:47:23,092::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-16::DEBUG::2014-08-11 14:47:23,093::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0024796 s, 183 kB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 14:47:23,095::task::993::Storage.TaskManager.Task::(_decref) Task=`3d874ddc-4f37-495c-9fb4-3da9f2785b41`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:47:23,096::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:23,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:23,099::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:23,101::__init__::232::IOProcess::(_processLogs) (5712) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:23,102::__init__::232::IOProcess::(_processLogs) DEBUG|(5712) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:23,103::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:23,104::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:23,104::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:23,104::__init__::232::IOProcess::(_processLogs) (5713) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:23,104::__init__::232::IOProcess::(_processLogs) (5713) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:23,105::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:23,106::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:23,106::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:23,106::__init__::232::IOProcess::(_processLogs) (5714) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:23,107::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5714) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:23,107::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:23,108::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:23,109::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:23,109::__init__::232::IOProcess::(_processLogs) (5715) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:23,110::__init__::232::IOProcess::(_processLogs) (5715) Queuing response >Dummy-33::DEBUG::2014-08-11 14:47:24,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:24,319::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131625 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:26,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:26,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132941 s, 77.0 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:47:27,384::task::595::Storage.TaskManager.Task::(_updateState) Task=`61f3b586-5825-4404-a7b6-62a0719254bd`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:27,385::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::INFO::2014-08-11 14:47:27,385::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:47:27,468::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0024796', 'lastCheck': '4.4', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:47:27,470::task::1191::Storage.TaskManager.Task::(prepare) Task=`61f3b586-5825-4404-a7b6-62a0719254bd`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0024796', 'lastCheck': '4.4', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:27,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:47:27,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`61f3b586-5825-4404-a7b6-62a0719254bd`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:27,472::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:27,474::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:47:27,475::task::993::Storage.TaskManager.Task::(_decref) Task=`61f3b586-5825-4404-a7b6-62a0719254bd`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:27,477::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:47:27,501::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:28,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:28,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132638 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:30,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:30,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133072 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:32,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:32,492::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:32,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:32,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:32,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303362 s, 33.8 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:47:33,109::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:47:33,112::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,112::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,113::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,116::__init__::232::IOProcess::(_processLogs) (5716) Got request for method 'glob' >Thread-103::DEBUG::2014-08-11 14:47:33,115::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:47:33,119::__init__::232::IOProcess::(_processLogs) (5716) Queuing response >Thread-103::DEBUG::2014-08-11 14:47:33,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b40f1f7-6d73-42c4-b96f-20fdf5143e6c`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:47:33,123::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 14:47:33,129::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:47:33,130::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,130::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,131::__init__::232::IOProcess::(_processLogs) (5717) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 14:47:33,131::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5717) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:33,132::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:47:33,133::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,133::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,133::__init__::232::IOProcess::(_processLogs) (5718) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:47:33,134::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5718) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:33,134::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:47:33,135::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,136::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,137::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,137::__init__::232::IOProcess::(_processLogs) (5719) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:33,137::__init__::232::IOProcess::(_processLogs) (5719) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:33,138::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,138::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,138::__init__::232::IOProcess::(_processLogs) (5720) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:47:33,139::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5720) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:33,139::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 14:47:33,140::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:47:33,140::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:47:33,141::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,142::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:47:33,143::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-103::INFO::2014-08-11 14:47:33,163::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:33,164::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b40f1f7-6d73-42c4-b96f-20fdf5143e6c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:33,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b40f1f7-6d73-42c4-b96f-20fdf5143e6c`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:33,165::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:33,165::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:47:33,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-103::DEBUG::2014-08-11 14:47:33,166::task::993::Storage.TaskManager.Task::(_decref) Task=`1b40f1f7-6d73-42c4-b96f-20fdf5143e6c`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:47:33,167::__init__::232::IOProcess::(_processLogs) (5721) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:33,169::__init__::232::IOProcess::(_processLogs) (5721) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:33,170::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00249529 s, 182 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:47:33,171::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,172::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,172::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,172::__init__::232::IOProcess::(_processLogs) (5722) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:33,173::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5722) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:33,174::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,175::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,176::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,176::__init__::232::IOProcess::(_processLogs) (5723) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:33,176::__init__::232::IOProcess::(_processLogs) (5723) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:33,176::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:33,177::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:33,178::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,178::__init__::232::IOProcess::(_processLogs) (5724) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:33,180::__init__::232::IOProcess::(_processLogs) (5724) Queuing response >Thread-34::DEBUG::2014-08-11 14:47:33,181::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 14:47:33,182::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-34::DEBUG::2014-08-11 14:47:33,183::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff4d4509-9dc8-473e-b660-fdc78bb61e47`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 14:47:33,184::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::INFO::2014-08-11 14:47:33,185::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 14:47:33,185::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:33,187::__init__::232::IOProcess::(_processLogs) (5725) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 14:47:33,189::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8815f8d5-b9ac-4077-a385-b28fe3e7a0e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:47:33,190::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:47:33,190::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:47:33,191::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8815f8d5-b9ac-4077-a385-b28fe3e7a0e7`::Granted request >Thread-14::DEBUG::2014-08-11 14:47:33,189::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5725) Queuing response >Thread-34::DEBUG::2014-08-11 14:47:33,194::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ff4d4509-9dc8-473e-b660-fdc78bb61e47`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:47:33,194::task::993::Storage.TaskManager.Task::(_decref) Task=`ff4d4509-9dc8-473e-b660-fdc78bb61e47`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:47:33,214::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860357914624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:33,214::task::1191::Storage.TaskManager.Task::(prepare) Task=`ff4d4509-9dc8-473e-b660-fdc78bb61e47`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860357914624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:33,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff4d4509-9dc8-473e-b660-fdc78bb61e47`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:47:33,215::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:47:33,215::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:47:33,215::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:47:33,216::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:47:33,216::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:47:33,216::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:47:33,216::task::993::Storage.TaskManager.Task::(_decref) Task=`ff4d4509-9dc8-473e-b660-fdc78bb61e47`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:47:34,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:34,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132341 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:36,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:36,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132375 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:37,597::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:37,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:37,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:38,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:38,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132807 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:41,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:41,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137446 s, 74.5 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:47:42,596::task::595::Storage.TaskManager.Task::(_updateState) Task=`420a0a0a-00e5-4193-adda-e618ab5d3db8`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:47:42,596::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:47:42,597::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00249529', 'lastCheck': '9.4', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:47:42,597::task::1191::Storage.TaskManager.Task::(prepare) Task=`420a0a0a-00e5-4193-adda-e618ab5d3db8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00249529', 'lastCheck': '9.4', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:47:42,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`420a0a0a-00e5-4193-adda-e618ab5d3db8`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:42,597::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:42,598::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:47:42,598::task::993::Storage.TaskManager.Task::(_decref) Task=`420a0a0a-00e5-4193-adda-e618ab5d3db8`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:47:42,617::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:42,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:42,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:42,677::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:43,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:43,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131964 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:47:43,191::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:43,191::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:43,192::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:43,192::__init__::232::IOProcess::(_processLogs) (5726) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:43,192::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5726) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:43,193::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:47:43,212::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000512329 s, 888 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:47:43,213::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:43,214::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:43,214::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:43,214::__init__::232::IOProcess::(_processLogs) (5727) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:43,215::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5727) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:43,216::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:43,216::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:43,216::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:43,217::__init__::232::IOProcess::(_processLogs) (5728) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:43,217::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5728) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:43,218::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:43,218::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:43,218::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:43,218::__init__::232::IOProcess::(_processLogs) (5729) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:43,219::__init__::232::IOProcess::(_processLogs) (5729) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:43,219::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:43,220::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:43,220::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:43,222::__init__::232::IOProcess::(_processLogs) (5730) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:43,222::__init__::232::IOProcess::(_processLogs) (5730) Queuing response >Thread-103::DEBUG::2014-08-11 14:47:43,228::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-103::DEBUG::2014-08-11 14:47:43,229::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef45493f-f0ab-4414-bf87-3b25343d6312`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:47:43,229::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:47:43,242::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:43,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`ef45493f-f0ab-4414-bf87-3b25343d6312`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:43,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef45493f-f0ab-4414-bf87-3b25343d6312`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:43,243::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:43,243::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:47:43,243::task::993::Storage.TaskManager.Task::(_decref) Task=`ef45493f-f0ab-4414-bf87-3b25343d6312`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:47:43,262::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-34::DEBUG::2014-08-11 14:47:43,263::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cb5031a-1a24-485c-9610-5da649573f28`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:47:43,263::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:47:43,264::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9addf346-4cb2-4ad9-b253-ce334109161c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:47:43,264::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:47:43,264::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:47:43,265::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9addf346-4cb2-4ad9-b253-ce334109161c`::Granted request >Thread-34::DEBUG::2014-08-11 14:47:43,265::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6cb5031a-1a24-485c-9610-5da649573f28`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:47:43,265::task::993::Storage.TaskManager.Task::(_decref) Task=`6cb5031a-1a24-485c-9610-5da649573f28`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:47:43,304::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860325081088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:43,304::task::1191::Storage.TaskManager.Task::(prepare) Task=`6cb5031a-1a24-485c-9610-5da649573f28`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860325081088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:43,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cb5031a-1a24-485c-9610-5da649573f28`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:47:43,304::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:47:43,305::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:47:43,305::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:47:43,305::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:47:43,305::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:47:43,306::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:47:43,306::task::993::Storage.TaskManager.Task::(_decref) Task=`6cb5031a-1a24-485c-9610-5da649573f28`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:47:45,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:45,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131977 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:47,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:47,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132831 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:47,681::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:47,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:47,769::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:49,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:49,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013246 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:51,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:51,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013258 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:52,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:52,777::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:52,869::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:47:53,222::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:53,223::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:53,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:53,223::__init__::232::IOProcess::(_processLogs) (5731) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:53,223::__init__::232::IOProcess::(_processLogs) (5731) Queuing response >Thread-16::DEBUG::2014-08-11 14:47:53,224::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:47:53,243::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418925 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:47:53,244::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:53,245::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:53,245::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:53,245::__init__::232::IOProcess::(_processLogs) (5732) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:47:53,246::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5732) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:53,247::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:53,248::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:53,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:53,249::__init__::232::IOProcess::(_processLogs) (5733) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:53,249::__init__::232::IOProcess::(_processLogs) (5733) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:53,249::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:53,250::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:53,250::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:53,251::__init__::232::IOProcess::(_processLogs) (5734) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:53,251::__init__::232::IOProcess::(_processLogs) (5734) Queuing response >Thread-14::DEBUG::2014-08-11 14:47:53,251::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:47:53,252::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:47:53,252::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:47:53,253::__init__::232::IOProcess::(_processLogs) (5735) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:47:53,254::__init__::232::IOProcess::(_processLogs) (5735) Queuing response >Thread-103::DEBUG::2014-08-11 14:47:53,318::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-103::DEBUG::2014-08-11 14:47:53,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`6297914d-d99b-479d-baed-c4f5fee1cc20`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:47:53,319::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:47:53,332::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:53,333::task::1191::Storage.TaskManager.Task::(prepare) Task=`6297914d-d99b-479d-baed-c4f5fee1cc20`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:47:53,334::task::595::Storage.TaskManager.Task::(_updateState) Task=`6297914d-d99b-479d-baed-c4f5fee1cc20`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:53,335::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:53,336::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:47:53,338::task::993::Storage.TaskManager.Task::(_decref) Task=`6297914d-d99b-479d-baed-c4f5fee1cc20`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:47:53,348::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 14:47:53,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`7cbc5e6e-c24f-4e71-8d52-7256dea84e2c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:47:53,349::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:47:53,350::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`985e6d65-1eba-4a22-a5bd-a29805ae7864`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:47:53,350::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:47:53,350::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:47:53,351::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`985e6d65-1eba-4a22-a5bd-a29805ae7864`::Granted request >Thread-34::DEBUG::2014-08-11 14:47:53,351::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7cbc5e6e-c24f-4e71-8d52-7256dea84e2c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Dummy-33::DEBUG::2014-08-11 14:47:53,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::DEBUG::2014-08-11 14:47:53,353::task::993::Storage.TaskManager.Task::(_decref) Task=`7cbc5e6e-c24f-4e71-8d52-7256dea84e2c`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:47:53,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021036 s, 48.7 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 14:47:53,413::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860325081088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:53,415::task::1191::Storage.TaskManager.Task::(prepare) Task=`7cbc5e6e-c24f-4e71-8d52-7256dea84e2c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860325081088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:47:53,416::task::595::Storage.TaskManager.Task::(_updateState) Task=`7cbc5e6e-c24f-4e71-8d52-7256dea84e2c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:47:53,417::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:47:53,419::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:47:53,421::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:47:53,423::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:47:53,424::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:47:53,425::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:47:53,427::task::993::Storage.TaskManager.Task::(_decref) Task=`7cbc5e6e-c24f-4e71-8d52-7256dea84e2c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:47:55,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:55,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132518 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:47:57,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:57,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131761 s, 77.7 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:47:57,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0e14091-21c3-43af-b642-a186e25f1dc1`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:47:57,722::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:47:57,722::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418925', 'lastCheck': '4.5', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:47:57,722::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0e14091-21c3-43af-b642-a186e25f1dc1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418925', 'lastCheck': '4.5', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:47:57,723::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0e14091-21c3-43af-b642-a186e25f1dc1`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:47:57,723::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:47:57,723::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:47:57,723::task::993::Storage.TaskManager.Task::(_decref) Task=`e0e14091-21c3-43af-b642-a186e25f1dc1`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:47:57,742::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:57,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:57,876::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:47:57,876::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:47:59,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:47:59,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131924 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:01,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:01,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132745 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:02,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:02,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:02,970::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:48:03,254::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:48:03,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:48:03,255::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:03,255::__init__::232::IOProcess::(_processLogs) (5736) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:48:03,255::__init__::232::IOProcess::(_processLogs) (5736) Queuing response >Thread-16::DEBUG::2014-08-11 14:48:03,256::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:48:03,274::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417954 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:48:03,276::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:03,276::__init__::232::IOProcess::(_processLogs) (5737) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:48:03,277::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:03,277::__init__::232::IOProcess::(_processLogs) (5738) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:03,278::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:03,279::__init__::232::IOProcess::(_processLogs) (5739) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:03,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:03,280::__init__::232::IOProcess::(_processLogs) (5740) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:03,280::__init__::232::IOProcess::(_processLogs) (5740) Queuing response >Thread-103::DEBUG::2014-08-11 14:48:03,443::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:48:03,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`38cb2c9b-984f-41af-9ae8-68b14ef45eed`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:03,445::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:48:03,458::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:03,459::task::1191::Storage.TaskManager.Task::(prepare) Task=`38cb2c9b-984f-41af-9ae8-68b14ef45eed`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:03,459::task::595::Storage.TaskManager.Task::(_updateState) Task=`38cb2c9b-984f-41af-9ae8-68b14ef45eed`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:03,459::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:03,459::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:03,459::task::993::Storage.TaskManager.Task::(_decref) Task=`38cb2c9b-984f-41af-9ae8-68b14ef45eed`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:48:03,469::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:48:03,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`b25097d9-3ca1-44bb-8de3-8c97e3335d72`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:48:03,470::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:48:03,471::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a8bcf8c6-cc7d-4b19-9a5a-a2784139e687`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:48:03,471::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:48:03,472::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:48:03,472::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a8bcf8c6-cc7d-4b19-9a5a-a2784139e687`::Granted request >Thread-34::DEBUG::2014-08-11 14:48:03,472::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b25097d9-3ca1-44bb-8de3-8c97e3335d72`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:48:03,473::task::993::Storage.TaskManager.Task::(_decref) Task=`b25097d9-3ca1-44bb-8de3-8c97e3335d72`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:48:03,492::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860325081088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:03,493::task::1191::Storage.TaskManager.Task::(prepare) Task=`b25097d9-3ca1-44bb-8de3-8c97e3335d72`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860325081088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:03,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`b25097d9-3ca1-44bb-8de3-8c97e3335d72`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:48:03,493::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:48:03,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:48:03,494::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:48:03,494::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:48:03,494::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:48:03,494::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:48:03,495::task::993::Storage.TaskManager.Task::(_decref) Task=`b25097d9-3ca1-44bb-8de3-8c97e3335d72`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:48:03,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:03,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013193 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:05,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:05,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132035 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:07,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:07,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132319 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:07,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:07,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:08,070::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:48:09,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:09,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132974 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:11,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:11,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279223 s, 36.7 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:48:12,832::task::595::Storage.TaskManager.Task::(_updateState) Task=`83b1b059-8bab-4b5b-a7ae-f6f3b658e067`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:12,832::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:48:12,833::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417954', 'lastCheck': '9.6', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:12,833::task::1191::Storage.TaskManager.Task::(prepare) Task=`83b1b059-8bab-4b5b-a7ae-f6f3b658e067`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417954', 'lastCheck': '9.6', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:12,833::task::595::Storage.TaskManager.Task::(_updateState) Task=`83b1b059-8bab-4b5b-a7ae-f6f3b658e067`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:12,833::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:12,834::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:12,834::task::993::Storage.TaskManager.Task::(_decref) Task=`83b1b059-8bab-4b5b-a7ae-f6f3b658e067`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:48:12,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:13,074::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:13,075::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:13,076::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:48:13,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`492efc50-609c-4b84-9b69-5dfed705d580`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:48:13,170::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:48:13,171::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:13,171::__init__::232::IOProcess::(_processLogs) (5741) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:48:13,173::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:13,174::__init__::232::IOProcess::(_processLogs) (5742) Got request for method 'stat' >Thread-55::INFO::2014-08-11 14:48:13,174::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:48:13,174::task::1191::Storage.TaskManager.Task::(prepare) Task=`492efc50-609c-4b84-9b69-5dfed705d580`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:48:13,175::task::595::Storage.TaskManager.Task::(_updateState) Task=`492efc50-609c-4b84-9b69-5dfed705d580`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:48:13,175::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:48:13,175::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:48:13,175::task::993::Storage.TaskManager.Task::(_decref) Task=`492efc50-609c-4b84-9b69-5dfed705d580`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:48:13,283::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:48:13,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:13,285::__init__::232::IOProcess::(_processLogs) (5743) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:48:13,289::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:48:13,310::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000515726 s, 882 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:48:13,314::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:48:13,315::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:13,316::__init__::232::IOProcess::(_processLogs) (5744) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:48:13,320::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:48:13,322::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:13,323::__init__::232::IOProcess::(_processLogs) (5745) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:13,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:48:13,330::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:13,332::__init__::232::IOProcess::(_processLogs) (5746) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:13,333::__init__::232::IOProcess::(_processLogs) (5746) Queuing response >Thread-14::DEBUG::2014-08-11 14:48:13,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:48:13,337::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:13,338::__init__::232::IOProcess::(_processLogs) (5747) Got request for method 'access' >Thread-103::DEBUG::2014-08-11 14:48:13,508::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:48:13,509::task::595::Storage.TaskManager.Task::(_updateState) Task=`c23846d0-e343-46be-a4d1-f7d44ae420dc`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:13,510::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:48:13,522::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:13,523::task::1191::Storage.TaskManager.Task::(prepare) Task=`c23846d0-e343-46be-a4d1-f7d44ae420dc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:13,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`c23846d0-e343-46be-a4d1-f7d44ae420dc`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:13,523::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:13,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:13,523::task::993::Storage.TaskManager.Task::(_decref) Task=`c23846d0-e343-46be-a4d1-f7d44ae420dc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:48:13,535::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:48:13,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`f8d811c1-11ec-46a1-8648-f8034e9848e0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:48:13,536::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:48:13,537::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`65b5e5f2-cd6e-426d-94ac-478e85cc0714`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:48:13,538::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:48:13,538::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:48:13,538::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`65b5e5f2-cd6e-426d-94ac-478e85cc0714`::Granted request >Thread-34::DEBUG::2014-08-11 14:48:13,539::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f8d811c1-11ec-46a1-8648-f8034e9848e0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:48:13,539::task::993::Storage.TaskManager.Task::(_decref) Task=`f8d811c1-11ec-46a1-8648-f8034e9848e0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:48:13,559::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860314464256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:13,559::task::1191::Storage.TaskManager.Task::(prepare) Task=`f8d811c1-11ec-46a1-8648-f8034e9848e0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860314464256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:13,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`f8d811c1-11ec-46a1-8648-f8034e9848e0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:48:13,560::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:48:13,560::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:48:13,560::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:48:13,561::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:48:13,561::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:48:13,561::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:48:13,561::task::993::Storage.TaskManager.Task::(_decref) Task=`f8d811c1-11ec-46a1-8648-f8034e9848e0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:48:13,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:15,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.4254 s, 718 kB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:17,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:17,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017688 s, 57.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:18,084::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:18,085::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:18,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:48:19,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:19,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132103 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:21,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:21,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132505 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:23,093::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:23,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:23,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:48:23,342::__init__::232::IOProcess::(_processLogs) (5748) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:48:23,343::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:48:23,361::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415419 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:48:23,363::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:23,363::__init__::232::IOProcess::(_processLogs) (5749) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:48:23,364::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:23,365::__init__::232::IOProcess::(_processLogs) (5750) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:23,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:23,366::__init__::232::IOProcess::(_processLogs) (5751) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:23,367::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:23,367::__init__::232::IOProcess::(_processLogs) (5752) Got request for method 'access' >Thread-103::DEBUG::2014-08-11 14:48:23,573::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:48:23,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4930612-89a3-4ca7-a6d7-7c22144fd878`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:23,574::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 14:48:23,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-103::INFO::2014-08-11 14:48:23,604::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:23,605::task::1191::Storage.TaskManager.Task::(prepare) Task=`b4930612-89a3-4ca7-a6d7-7c22144fd878`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:23,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4930612-89a3-4ca7-a6d7-7c22144fd878`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:23,605::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:23,606::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:23,606::task::993::Storage.TaskManager.Task::(_decref) Task=`b4930612-89a3-4ca7-a6d7-7c22144fd878`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:48:23,618::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:48:23,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cde2bac-198c-40b1-b634-fa840b317f6c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:48:23,619::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:48:23,620::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`41159590-fe8c-47b1-9e1c-df9846fd5a41`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:48:23,620::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:48:23,621::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:48:23,621::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`41159590-fe8c-47b1-9e1c-df9846fd5a41`::Granted request >Thread-34::DEBUG::2014-08-11 14:48:23,621::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1cde2bac-198c-40b1-b634-fa840b317f6c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:48:23,622::task::993::Storage.TaskManager.Task::(_decref) Task=`1cde2bac-198c-40b1-b634-fa840b317f6c`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:48:23,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0361038 s, 28.4 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 14:48:23,671::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860280188928', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:23,672::task::1191::Storage.TaskManager.Task::(prepare) Task=`1cde2bac-198c-40b1-b634-fa840b317f6c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860280188928', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:23,673::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cde2bac-198c-40b1-b634-fa840b317f6c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:48:23,674::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:48:23,675::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:48:23,677::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:48:23,678::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:48:23,679::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:48:23,680::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:48:23,681::task::993::Storage.TaskManager.Task::(_decref) Task=`1cde2bac-198c-40b1-b634-fa840b317f6c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:48:25,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:25,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132347 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:27,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:27,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013161 s, 77.8 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:48:27,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7e4636d-251a-44eb-82e7-dc968546c2d3`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:27,944::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:48:27,945::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415419', 'lastCheck': '4.6', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:27,945::task::1191::Storage.TaskManager.Task::(prepare) Task=`f7e4636d-251a-44eb-82e7-dc968546c2d3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415419', 'lastCheck': '4.6', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:27,945::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7e4636d-251a-44eb-82e7-dc968546c2d3`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:27,945::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:27,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:27,946::task::993::Storage.TaskManager.Task::(_decref) Task=`f7e4636d-251a-44eb-82e7-dc968546c2d3`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:48:27,965::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:28,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:28,103::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:28,104::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:48:29,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:29,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132245 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:31,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:31,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133961 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:33,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:33,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:33,112::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:48:33,369::__init__::232::IOProcess::(_processLogs) (5753) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:48:33,370::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:48:33,389::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420387 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:48:33,391::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:33,391::__init__::232::IOProcess::(_processLogs) (5754) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:48:33,392::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:33,393::__init__::232::IOProcess::(_processLogs) (5755) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:33,393::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:33,394::__init__::232::IOProcess::(_processLogs) (5756) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:33,395::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:33,395::__init__::232::IOProcess::(_processLogs) (5757) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:33,395::__init__::232::IOProcess::(_processLogs) (5757) Queuing response >Thread-103::DEBUG::2014-08-11 14:48:33,696::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:48:33,696::task::595::Storage.TaskManager.Task::(_updateState) Task=`2af10281-0211-4bdd-bdec-029d016a0713`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:33,697::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:48:33,709::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:33,710::task::1191::Storage.TaskManager.Task::(prepare) Task=`2af10281-0211-4bdd-bdec-029d016a0713`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:33,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`2af10281-0211-4bdd-bdec-029d016a0713`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:33,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:33,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:33,711::task::993::Storage.TaskManager.Task::(_decref) Task=`2af10281-0211-4bdd-bdec-029d016a0713`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:48:33,721::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:48:33,722::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb985f2e-0d6b-4618-aa84-6e1ca54b45e4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:48:33,722::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:48:33,723::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0044c116-9d20-4021-8920-3b63292eaa0f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:48:33,723::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:48:33,724::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:48:33,724::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0044c116-9d20-4021-8920-3b63292eaa0f`::Granted request >Thread-34::DEBUG::2014-08-11 14:48:33,724::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bb985f2e-0d6b-4618-aa84-6e1ca54b45e4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:48:33,724::task::993::Storage.TaskManager.Task::(_decref) Task=`bb985f2e-0d6b-4618-aa84-6e1ca54b45e4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:48:33,744::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860277633024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:33,744::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb985f2e-0d6b-4618-aa84-6e1ca54b45e4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860277633024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:33,745::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb985f2e-0d6b-4618-aa84-6e1ca54b45e4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:48:33,745::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:48:33,745::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:48:33,746::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:48:33,746::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:48:33,746::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:48:33,746::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:48:33,747::task::993::Storage.TaskManager.Task::(_decref) Task=`bb985f2e-0d6b-4618-aa84-6e1ca54b45e4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:48:33,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:33,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132296 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:35,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:35,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133551 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:38,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:38,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131992 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:38,119::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:38,120::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:38,121::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:48:40,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:40,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132187 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:42,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:42,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013166 s, 77.8 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:48:43,058::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9e7d1b5-1a3d-4e93-a320-b65cb05785b1`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:43,058::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:48:43,070::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420387', 'lastCheck': '9.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:43,070::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9e7d1b5-1a3d-4e93-a320-b65cb05785b1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420387', 'lastCheck': '9.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:43,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9e7d1b5-1a3d-4e93-a320-b65cb05785b1`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:43,070::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:43,070::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:43,071::task::993::Storage.TaskManager.Task::(_decref) Task=`c9e7d1b5-1a3d-4e93-a320-b65cb05785b1`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:48:43,090::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:43,124::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:43,125::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:43,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:48:43,397::__init__::232::IOProcess::(_processLogs) (5758) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:48:43,398::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:48:43,417::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418179 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:48:43,418::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:43,418::__init__::232::IOProcess::(_processLogs) (5759) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:48:43,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:43,420::__init__::232::IOProcess::(_processLogs) (5760) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:43,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:43,421::__init__::232::IOProcess::(_processLogs) (5761) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:43,421::__init__::232::IOProcess::(_processLogs) (5761) Queuing response >Thread-14::DEBUG::2014-08-11 14:48:43,422::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:43,422::__init__::232::IOProcess::(_processLogs) (5762) Got request for method 'access' >Thread-103::DEBUG::2014-08-11 14:48:43,760::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:48:43,760::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0cecc56-7105-4fe4-ab65-944a6690d179`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:43,761::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:48:43,773::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:43,774::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0cecc56-7105-4fe4-ab65-944a6690d179`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:43,774::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0cecc56-7105-4fe4-ab65-944a6690d179`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:43,774::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:43,774::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:43,775::task::993::Storage.TaskManager.Task::(_decref) Task=`e0cecc56-7105-4fe4-ab65-944a6690d179`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:48:43,786::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:48:43,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7d6a8f1-1c99-43c9-8ac0-5d6d2480236b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:48:43,787::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:48:43,787::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed990288-412c-4592-ba3a-9f588774cb02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:48:43,788::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:48:43,788::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:48:43,788::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed990288-412c-4592-ba3a-9f588774cb02`::Granted request >Thread-34::DEBUG::2014-08-11 14:48:43,789::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c7d6a8f1-1c99-43c9-8ac0-5d6d2480236b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:48:43,789::task::993::Storage.TaskManager.Task::(_decref) Task=`c7d6a8f1-1c99-43c9-8ac0-5d6d2480236b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:48:43,809::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860275404800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:43,809::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7d6a8f1-1c99-43c9-8ac0-5d6d2480236b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860275404800', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:43,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7d6a8f1-1c99-43c9-8ac0-5d6d2480236b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:48:43,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:48:43,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:48:43,810::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:48:43,811::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:48:43,811::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:48:43,811::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:48:43,811::task::993::Storage.TaskManager.Task::(_decref) Task=`c7d6a8f1-1c99-43c9-8ac0-5d6d2480236b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:48:44,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:44,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135364 s, 75.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:46,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:46,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132206 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:48,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:48,134::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:48,170::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:48:48,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:48,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131792 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:50,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:50,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013597 s, 75.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:52,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:52,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013194 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:53,177::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:53,179::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:53,270::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:48:53,424::__init__::232::IOProcess::(_processLogs) (5763) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:48:53,425::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:48:53,445::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425236 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:48:53,446::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:53,447::__init__::232::IOProcess::(_processLogs) (5764) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:48:53,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:53,448::__init__::232::IOProcess::(_processLogs) (5765) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:53,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:53,450::__init__::232::IOProcess::(_processLogs) (5766) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:53,450::__init__::232::IOProcess::(_processLogs) (5766) Queuing response >Thread-14::DEBUG::2014-08-11 14:48:53,451::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:48:53,452::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:48:53,452::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:48:53,453::__init__::232::IOProcess::(_processLogs) (5767) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:48:53,453::__init__::232::IOProcess::(_processLogs) (5767) Queuing response >Thread-103::DEBUG::2014-08-11 14:48:53,824::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-103::DEBUG::2014-08-11 14:48:53,825::task::595::Storage.TaskManager.Task::(_updateState) Task=`70d4e43c-f76e-4152-8da2-764aa412a12d`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:53,825::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:48:53,838::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:53,839::task::1191::Storage.TaskManager.Task::(prepare) Task=`70d4e43c-f76e-4152-8da2-764aa412a12d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:48:53,839::task::595::Storage.TaskManager.Task::(_updateState) Task=`70d4e43c-f76e-4152-8da2-764aa412a12d`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:53,839::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:53,839::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:53,840::task::993::Storage.TaskManager.Task::(_decref) Task=`70d4e43c-f76e-4152-8da2-764aa412a12d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:48:53,851::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 14:48:53,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cc08a8e-1d9e-4a20-a00c-3b5f9ee33758`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:48:53,852::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:48:53,853::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4f59344f-ff53-4ddf-bdef-974a9c34cfa3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:48:53,853::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:48:53,853::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:48:53,854::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4f59344f-ff53-4ddf-bdef-974a9c34cfa3`::Granted request >Thread-34::DEBUG::2014-08-11 14:48:53,854::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3cc08a8e-1d9e-4a20-a00c-3b5f9ee33758`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:48:53,855::task::993::Storage.TaskManager.Task::(_decref) Task=`3cc08a8e-1d9e-4a20-a00c-3b5f9ee33758`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:48:53,874::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:53,874::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cc08a8e-1d9e-4a20-a00c-3b5f9ee33758`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:48:53,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cc08a8e-1d9e-4a20-a00c-3b5f9ee33758`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:48:53,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:48:53,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:48:53,876::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:48:53,876::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:48:53,876::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:48:53,876::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:48:53,877::task::993::Storage.TaskManager.Task::(_decref) Task=`3cc08a8e-1d9e-4a20-a00c-3b5f9ee33758`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:48:54,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:54,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132816 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:48:56,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:56,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0226521 s, 45.2 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:48:58,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`84fb7cb8-4610-422e-baa3-f19c535dd5de`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:48:58,178::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:48:58,179::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425236', 'lastCheck': '4.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:58,269::task::1191::Storage.TaskManager.Task::(prepare) Task=`84fb7cb8-4610-422e-baa3-f19c535dd5de`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425236', 'lastCheck': '4.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:48:58,270::task::595::Storage.TaskManager.Task::(_updateState) Task=`84fb7cb8-4610-422e-baa3-f19c535dd5de`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:48:58,270::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:48:58,270::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:48:58,270::task::993::Storage.TaskManager.Task::(_decref) Task=`84fb7cb8-4610-422e-baa3-f19c535dd5de`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:58,277::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:58,279::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:48:58,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:48:58,297::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:48:58,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:48:58,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132662 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:00,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:00,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132531 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:02,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:02,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261355 s, 39.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:03,284::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:03,286::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:03,371::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:49:03,454::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:03,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:03,455::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:49:03,456::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:49:03,456::__init__::232::IOProcess::(_processLogs) (5768) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:03,474::__init__::232::IOProcess::(_processLogs) (5768) Queuing response >Thread-16::DEBUG::2014-08-11 14:49:03,476::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000517369 s, 879 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:03,477::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:03,477::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:03,477::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:03,478::__init__::232::IOProcess::(_processLogs) (5769) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:03,478::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5769) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:03,480::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:03,481::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:03,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:03,481::__init__::232::IOProcess::(_processLogs) (5770) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:03,482::__init__::232::IOProcess::(_processLogs) (5770) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:03,482::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:03,483::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:03,483::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:03,484::__init__::232::IOProcess::(_processLogs) (5771) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:03,484::__init__::232::IOProcess::(_processLogs) (5771) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:03,484::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:03,485::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:03,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:03,486::__init__::232::IOProcess::(_processLogs) (5772) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:03,486::__init__::232::IOProcess::(_processLogs) (5772) Queuing response >Thread-103::DEBUG::2014-08-11 14:49:03,888::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:49:03,888::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1008068-aa23-492c-8934-da0a5aeb8d7e`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:03,889::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:49:03,902::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:03,903::task::1191::Storage.TaskManager.Task::(prepare) Task=`c1008068-aa23-492c-8934-da0a5aeb8d7e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:03,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1008068-aa23-492c-8934-da0a5aeb8d7e`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:03,903::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:03,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:03,903::task::993::Storage.TaskManager.Task::(_decref) Task=`c1008068-aa23-492c-8934-da0a5aeb8d7e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:49:03,914::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:49:03,915::task::595::Storage.TaskManager.Task::(_updateState) Task=`715d0261-d999-4869-b53a-a576cd43a519`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:49:03,915::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:49:03,916::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1f3c1530-2e18-4301-bafc-f47410324af3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:49:03,916::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:49:03,916::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:49:03,917::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1f3c1530-2e18-4301-bafc-f47410324af3`::Granted request >Thread-34::DEBUG::2014-08-11 14:49:03,917::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`715d0261-d999-4869-b53a-a576cd43a519`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:49:03,918::task::993::Storage.TaskManager.Task::(_decref) Task=`715d0261-d999-4869-b53a-a576cd43a519`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:49:03,937::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:03,938::task::1191::Storage.TaskManager.Task::(prepare) Task=`715d0261-d999-4869-b53a-a576cd43a519`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:03,938::task::595::Storage.TaskManager.Task::(_updateState) Task=`715d0261-d999-4869-b53a-a576cd43a519`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:49:03,938::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:49:03,938::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:49:03,939::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:49:03,939::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:49:03,939::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:49:03,939::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:49:03,940::task::993::Storage.TaskManager.Task::(_decref) Task=`715d0261-d999-4869-b53a-a576cd43a519`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:49:04,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:04,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132088 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:06,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:07,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131616 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:08,378::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:08,380::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:08,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:49:09,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:09,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132462 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:11,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:11,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132773 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:13,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:13,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131755 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:49:13,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`28fb90c8-6e1b-4c87-91b8-ce6bd30a6a0e`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:49:13,293::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:49:13,294::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:13,295::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:13,295::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:13,295::__init__::232::IOProcess::(_processLogs) (5773) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:49:13,296::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5773) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:13,297::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:13,298::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:49:13,298::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:49:13,299::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:49:13,299::task::1191::Storage.TaskManager.Task::(prepare) Task=`28fb90c8-6e1b-4c87-91b8-ce6bd30a6a0e`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:49:13,300::__init__::232::IOProcess::(_processLogs) (5774) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:49:13,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`28fb90c8-6e1b-4c87-91b8-ce6bd30a6a0e`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:49:13,301::__init__::232::IOProcess::(_processLogs) (5774) Queuing response >Thread-55::DEBUG::2014-08-11 14:49:13,302::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:49:13,303::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:49:13,303::task::993::Storage.TaskManager.Task::(_decref) Task=`28fb90c8-6e1b-4c87-91b8-ce6bd30a6a0e`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:49:13,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc706c54-3bdd-4e9c-9738-c3d8b82147b5`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:13,394::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:49:13,472::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000517369', 'lastCheck': '10.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:49:13,472::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc706c54-3bdd-4e9c-9738-c3d8b82147b5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000517369', 'lastCheck': '10.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:49:13,472::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc706c54-3bdd-4e9c-9738-c3d8b82147b5`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:13,472::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:13,472::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:13,473::task::993::Storage.TaskManager.Task::(_decref) Task=`fc706c54-3bdd-4e9c-9738-c3d8b82147b5`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:13,479::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:13,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:13,483::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:49:13,487::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:13,487::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:13,487::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:13,488::__init__::232::IOProcess::(_processLogs) (5775) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:13,488::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5775) Queuing response >Thread-16::DEBUG::2014-08-11 14:49:13,489::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-103::DEBUG::2014-08-11 14:49:13,509::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:49:13,513::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000587339 s, 775 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:13,514::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:13,515::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:13,515::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:13,515::__init__::232::IOProcess::(_processLogs) (5776) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:13,516::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5776) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:13,517::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:13,518::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:13,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:13,518::__init__::232::IOProcess::(_processLogs) (5777) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:13,519::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5777) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:13,520::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:13,520::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:13,521::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:13,521::__init__::232::IOProcess::(_processLogs) (5778) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:13,521::__init__::232::IOProcess::(_processLogs) (5778) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:13,522::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:13,522::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:13,522::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:13,523::__init__::232::IOProcess::(_processLogs) (5779) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:13,523::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5779) Queuing response >Thread-103::DEBUG::2014-08-11 14:49:13,951::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:49:13,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ed1af44-cfe2-4758-bae0-aa79054374de`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:13,952::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:49:13,964::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:13,965::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ed1af44-cfe2-4758-bae0-aa79054374de`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:13,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ed1af44-cfe2-4758-bae0-aa79054374de`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:13,965::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:13,966::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:13,966::task::993::Storage.TaskManager.Task::(_decref) Task=`2ed1af44-cfe2-4758-bae0-aa79054374de`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:49:13,976::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:49:13,977::task::595::Storage.TaskManager.Task::(_updateState) Task=`709d89c0-ef51-4f51-9b4f-b717a2470496`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:49:13,977::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:49:13,978::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`05dd07a2-2709-4857-a94d-1b162d3bdb7c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:49:13,978::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:49:13,978::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:49:13,978::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`05dd07a2-2709-4857-a94d-1b162d3bdb7c`::Granted request >Thread-34::DEBUG::2014-08-11 14:49:13,980::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`709d89c0-ef51-4f51-9b4f-b717a2470496`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:49:13,980::task::993::Storage.TaskManager.Task::(_decref) Task=`709d89c0-ef51-4f51-9b4f-b717a2470496`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:49:15,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 14:49:15,378::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:15,379::task::1191::Storage.TaskManager.Task::(prepare) Task=`709d89c0-ef51-4f51-9b4f-b717a2470496`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:15,380::task::595::Storage.TaskManager.Task::(_updateState) Task=`709d89c0-ef51-4f51-9b4f-b717a2470496`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:49:15,380::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:49:15,381::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:49:15,382::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:49:15,383::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:49:15,384::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:49:15,384::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:49:15,385::task::993::Storage.TaskManager.Task::(_decref) Task=`709d89c0-ef51-4f51-9b4f-b717a2470496`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:49:15,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013974 s, 73.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:17,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:17,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131364 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:18,490::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:18,491::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:18,572::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:49:19,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:19,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133652 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:21,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:21,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133062 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:23,525::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:23,525::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:23,572::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:23,573::__init__::232::IOProcess::(_processLogs) (5780) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:49:23,573::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:49:23,574::__init__::232::IOProcess::(_processLogs) (5780) Queuing response >Dummy-33::DEBUG::2014-08-11 14:49:23,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:23,597::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:49:23,615::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000636823 s, 714 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:23,618::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:23,618::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:23,619::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:23,620::__init__::232::IOProcess::(_processLogs) (5781) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:23,623::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:23,626::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:49:23,626::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5781) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:23,633::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:23,634::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:23,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:23,639::__init__::232::IOProcess::(_processLogs) (5782) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:23,640::__init__::232::IOProcess::(_processLogs) (5782) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:23,641::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:23,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:23,642::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:23,642::__init__::232::IOProcess::(_processLogs) (5783) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:23,644::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5783) Queuing response >Dummy-33::DEBUG::2014-08-11 14:49:23,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273674 s, 37.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:23,647::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:23,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:23,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:23,650::__init__::232::IOProcess::(_processLogs) (5784) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:23,651::__init__::232::IOProcess::(_processLogs) (5784) Queuing response >Thread-103::DEBUG::2014-08-11 14:49:25,400::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:49:25,401::task::595::Storage.TaskManager.Task::(_updateState) Task=`42c7d917-ff84-4a64-b5d7-6bb5916f8435`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:25,401::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:49:25,413::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:25,414::task::1191::Storage.TaskManager.Task::(prepare) Task=`42c7d917-ff84-4a64-b5d7-6bb5916f8435`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:25,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`42c7d917-ff84-4a64-b5d7-6bb5916f8435`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:25,414::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:25,415::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:25,415::task::993::Storage.TaskManager.Task::(_decref) Task=`42c7d917-ff84-4a64-b5d7-6bb5916f8435`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:49:25,425::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:49:25,426::task::595::Storage.TaskManager.Task::(_updateState) Task=`79b7674d-6dc7-498b-af9b-c5b13e49f2e9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:49:25,426::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:49:25,427::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`73f0339c-e20b-4085-96f7-d2db00e368a5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:49:25,427::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:49:25,427::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:49:25,428::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`73f0339c-e20b-4085-96f7-d2db00e368a5`::Granted request >Thread-34::DEBUG::2014-08-11 14:49:25,428::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`79b7674d-6dc7-498b-af9b-c5b13e49f2e9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:49:25,428::task::993::Storage.TaskManager.Task::(_decref) Task=`79b7674d-6dc7-498b-af9b-c5b13e49f2e9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:49:25,448::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:25,449::task::1191::Storage.TaskManager.Task::(prepare) Task=`79b7674d-6dc7-498b-af9b-c5b13e49f2e9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272783360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:25,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`79b7674d-6dc7-498b-af9b-c5b13e49f2e9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:49:25,449::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:49:25,449::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:49:25,450::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:49:25,450::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:49:25,450::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:49:25,450::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:49:25,451::task::993::Storage.TaskManager.Task::(_decref) Task=`79b7674d-6dc7-498b-af9b-c5b13e49f2e9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:49:25,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:25,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013195 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:27,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:27,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132192 s, 77.5 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:49:28,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fce18a4-3b99-4779-9fef-86817b306d41`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:28,616::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:28,634::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::INFO::2014-08-11 14:49:28,673::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000636823', 'lastCheck': '5.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:49:28,674::task::1191::Storage.TaskManager.Task::(prepare) Task=`8fce18a4-3b99-4779-9fef-86817b306d41`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000636823', 'lastCheck': '5.0', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:28,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:49:28,676::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fce18a4-3b99-4779-9fef-86817b306d41`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:28,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:28,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:28,680::task::993::Storage.TaskManager.Task::(_decref) Task=`8fce18a4-3b99-4779-9fef-86817b306d41`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:28,681::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:49:28,704::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:49:29,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:29,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132364 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:31,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:31,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013453 s, 76.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:33,654::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:33,655::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:33,673::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:49:33,674::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:49:33,675::__init__::232::IOProcess::(_processLogs) (5785) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:33,693::__init__::232::IOProcess::(_processLogs) (5785) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:33,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:33,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:49:33,697::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041189 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:33,700::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:33,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:33,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:49:33,703::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:33,704::__init__::232::IOProcess::(_processLogs) (5786) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:33,706::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5786) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:33,708::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:33,709::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:33,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:33,709::__init__::232::IOProcess::(_processLogs) (5787) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:33,710::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|(5787) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:33,711::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:33,711::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:33,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:33,712::__init__::232::IOProcess::(_processLogs) (5788) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:33,712::__init__::232::IOProcess::(_processLogs) (5788) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:33,713::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:33,714::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:33,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:33,715::__init__::232::IOProcess::(_processLogs) (5789) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:33,715::__init__::232::IOProcess::(_processLogs) (5789) Queuing response >Dummy-33::DEBUG::2014-08-11 14:49:33,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:33,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134773 s, 76.0 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:49:35,465::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:49:35,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc3284ff-f0d8-4437-b954-988e4eaa399c`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:35,466::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:49:35,479::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:35,479::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc3284ff-f0d8-4437-b954-988e4eaa399c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:35,479::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc3284ff-f0d8-4437-b954-988e4eaa399c`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:35,479::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:35,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:35,480::task::993::Storage.TaskManager.Task::(_decref) Task=`cc3284ff-f0d8-4437-b954-988e4eaa399c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:49:35,490::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:49:35,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`7606d60e-bd94-4edd-bc31-1ca9c0cd3da1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:49:35,491::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:49:35,492::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7dd3db35-619c-4eb6-b084-4ec5ac925efa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:49:35,492::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:49:35,492::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:49:35,492::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7dd3db35-619c-4eb6-b084-4ec5ac925efa`::Granted request >Thread-34::DEBUG::2014-08-11 14:49:35,493::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7606d60e-bd94-4edd-bc31-1ca9c0cd3da1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:49:35,493::task::993::Storage.TaskManager.Task::(_decref) Task=`7606d60e-bd94-4edd-bc31-1ca9c0cd3da1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:49:35,513::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272717824', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:35,513::task::1191::Storage.TaskManager.Task::(prepare) Task=`7606d60e-bd94-4edd-bc31-1ca9c0cd3da1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860272717824', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:35,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`7606d60e-bd94-4edd-bc31-1ca9c0cd3da1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:49:35,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:49:35,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:49:35,514::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:49:35,515::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:49:35,515::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:49:35,515::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:49:35,515::task::993::Storage.TaskManager.Task::(_decref) Task=`7606d60e-bd94-4edd-bc31-1ca9c0cd3da1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:49:35,968::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:36,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132057 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:38,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:38,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132188 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:38,715::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:38,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:38,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:49:40,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:40,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132636 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:42,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:42,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160492 s, 63.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:43,716::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:43,716::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:49:43,718::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:49:43,718::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:43,790::__init__::232::IOProcess::(_processLogs) (5790) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:43,793::__init__::232::IOProcess::(_processLogs) (5790) Queuing response >Thread-16::DEBUG::2014-08-11 14:49:43,794::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00106493 s, 427 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:43,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:49:43,797::__init__::232::IOProcess::(_processLogs) (5766) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:43,799::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:43,801::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:49:43,801::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:43,803::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:43,805::__init__::232::IOProcess::(_processLogs) (5791) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:43,807::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:43,808::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:49:43,809::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:43,810::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:43,811::__init__::232::IOProcess::(_processLogs) (5792) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:43,812::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|(5792) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:43,813::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:43,814::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:43,814::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:43,814::__init__::232::IOProcess::(_processLogs) (5793) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:43,815::__init__::232::IOProcess::(_processLogs) (5793) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:43,815::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:43,816::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:43,816::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:43,817::__init__::232::IOProcess::(_processLogs) (5794) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:43,817::__init__::232::IOProcess::(_processLogs) (5794) Queuing response >Thread-103::DEBUG::2014-08-11 14:49:43,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1be1f0f-5834-4dfb-b7b5-be3db2d2ca1f`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:43,825::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:49:43,825::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00106493', 'lastCheck': '0.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:49:43,825::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1be1f0f-5834-4dfb-b7b5-be3db2d2ca1f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00106493', 'lastCheck': '0.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:49:43,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1be1f0f-5834-4dfb-b7b5-be3db2d2ca1f`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:43,826::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:43,826::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:43,826::task::993::Storage.TaskManager.Task::(_decref) Task=`e1be1f0f-5834-4dfb-b7b5-be3db2d2ca1f`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:49:43,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:49:44,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:44,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131481 s, 77.9 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:49:45,529::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:49:45,530::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c042f48-cdb8-4d12-a3e6-03588fb31c2e`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:45,530::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:49:45,543::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:45,543::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c042f48-cdb8-4d12-a3e6-03588fb31c2e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:45,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c042f48-cdb8-4d12-a3e6-03588fb31c2e`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:45,544::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:45,544::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:45,544::task::993::Storage.TaskManager.Task::(_decref) Task=`8c042f48-cdb8-4d12-a3e6-03588fb31c2e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:49:45,555::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:49:45,555::task::595::Storage.TaskManager.Task::(_updateState) Task=`e05b00cd-f3b8-4e15-8764-8e1f9fced4d6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:49:45,556::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:49:45,556::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a542cc85-41e2-4997-9be6-4bfcb38b1933`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:49:45,557::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:49:45,557::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:49:45,557::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a542cc85-41e2-4997-9be6-4bfcb38b1933`::Granted request >Thread-34::DEBUG::2014-08-11 14:49:45,558::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e05b00cd-f3b8-4e15-8764-8e1f9fced4d6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:49:45,558::task::993::Storage.TaskManager.Task::(_decref) Task=`e05b00cd-f3b8-4e15-8764-8e1f9fced4d6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:49:45,578::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860247945216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:45,578::task::1191::Storage.TaskManager.Task::(prepare) Task=`e05b00cd-f3b8-4e15-8764-8e1f9fced4d6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860247945216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:45,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`e05b00cd-f3b8-4e15-8764-8e1f9fced4d6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:49:45,579::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:49:45,579::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:49:45,579::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:49:45,580::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:49:45,580::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:49:45,580::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:49:45,581::task::993::Storage.TaskManager.Task::(_decref) Task=`e05b00cd-f3b8-4e15-8764-8e1f9fced4d6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:49:46,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:46,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132214 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:48,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:48,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131639 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:48,817::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:48,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:48,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:49:50,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:50,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013457 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:52,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:52,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133763 s, 76.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:53,818::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:53,818::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:49:53,819::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:49:53,874::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:53,892::__init__::232::IOProcess::(_processLogs) (5795) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:53,893::__init__::232::IOProcess::(_processLogs) (5795) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:53,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:53,896::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:53,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:49:53,899::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000400648 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:49:53,905::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:53,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:53,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:53,911::__init__::232::IOProcess::(_processLogs) (5796) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:49:53,911::__init__::232::IOProcess::(_processLogs) (5796) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:53,915::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:53,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:53,917::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:53,918::__init__::232::IOProcess::(_processLogs) (5797) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:53,919::__init__::232::IOProcess::(_processLogs) (5797) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:53,922::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:53,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:53,924::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:53,925::__init__::232::IOProcess::(_processLogs) (5798) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:53,926::__init__::232::IOProcess::(_processLogs) (5798) Queuing response >Thread-14::DEBUG::2014-08-11 14:49:53,926::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:49:53,927::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:49:53,927::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:49:53,928::__init__::232::IOProcess::(_processLogs) (5799) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:49:53,928::__init__::232::IOProcess::(_processLogs) (5799) Queuing response >Dummy-33::DEBUG::2014-08-11 14:49:54,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:54,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132472 s, 77.3 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:49:55,594::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:49:55,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`f59ac405-b824-4137-9e52-d90565bea714`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:55,595::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:49:55,608::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:55,608::task::1191::Storage.TaskManager.Task::(prepare) Task=`f59ac405-b824-4137-9e52-d90565bea714`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:49:55,609::task::595::Storage.TaskManager.Task::(_updateState) Task=`f59ac405-b824-4137-9e52-d90565bea714`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:55,609::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:49:55,609::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:55,609::task::993::Storage.TaskManager.Task::(_decref) Task=`f59ac405-b824-4137-9e52-d90565bea714`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:49:55,624::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:49:55,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`3dc7630c-ca61-4d1f-88d3-cbccb74d81a9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:49:55,626::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:49:55,626::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce70a866-548c-4de2-8a37-ea07109f6d57`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:49:55,626::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:49:55,627::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:49:55,627::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce70a866-548c-4de2-8a37-ea07109f6d57`::Granted request >Thread-34::DEBUG::2014-08-11 14:49:55,628::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3dc7630c-ca61-4d1f-88d3-cbccb74d81a9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:49:55,628::task::993::Storage.TaskManager.Task::(_decref) Task=`3dc7630c-ca61-4d1f-88d3-cbccb74d81a9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:49:55,648::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860247945216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:55,648::task::1191::Storage.TaskManager.Task::(prepare) Task=`3dc7630c-ca61-4d1f-88d3-cbccb74d81a9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860247945216', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:49:55,648::task::595::Storage.TaskManager.Task::(_updateState) Task=`3dc7630c-ca61-4d1f-88d3-cbccb74d81a9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:49:55,649::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:49:55,649::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:49:55,649::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:49:55,649::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:49:55,650::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:49:55,650::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:49:55,650::task::993::Storage.TaskManager.Task::(_decref) Task=`3dc7630c-ca61-4d1f-88d3-cbccb74d81a9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:49:56,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:56,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134863 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:49:58,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:49:58,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132527 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:58,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:58,914::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:49:58,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fcdc00f-aec8-4f91-9345-76920174e3bc`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:49:58,974::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:49:58,976::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000400648', 'lastCheck': '5.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:49:58,977::task::1191::Storage.TaskManager.Task::(prepare) Task=`7fcdc00f-aec8-4f91-9345-76920174e3bc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000400648', 'lastCheck': '5.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:49:58,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fcdc00f-aec8-4f91-9345-76920174e3bc`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:49:58,979::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:49:58,979::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:49:58,980::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:49:58,981::task::993::Storage.TaskManager.Task::(_decref) Task=`7fcdc00f-aec8-4f91-9345-76920174e3bc`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:49:59,003::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:00,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:00,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132769 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:02,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:02,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0259683 s, 39.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:03,929::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:03,929::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:03,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:03,975::__init__::232::IOProcess::(_processLogs) (5800) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:03,977::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|(5800) Queuing response >Thread-16::DEBUG::2014-08-11 14:50:03,979::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:04,003::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 14:50:04,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000596751 s, 762 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:04,009::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:04,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:04,015::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:04,015::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:04,018::__init__::232::IOProcess::(_processLogs) (5801) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:04,020::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|(5801) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:04,023::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:04,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:04,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:04,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:04,027::__init__::232::IOProcess::(_processLogs) (5802) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:04,028::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|(5802) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:04,029::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:04,030::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:04,030::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:04,031::__init__::232::IOProcess::(_processLogs) (5803) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:04,031::__init__::232::IOProcess::(_processLogs) (5803) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:04,031::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:04,032::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:04,032::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:04,033::__init__::232::IOProcess::(_processLogs) (5804) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:04,034::__init__::232::IOProcess::(_processLogs) (5804) Queuing response >Dummy-33::DEBUG::2014-08-11 14:50:04,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:04,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133031 s, 77.0 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:50:05,664::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:50:05,665::task::595::Storage.TaskManager.Task::(_updateState) Task=`cac0b550-9b78-460a-8a59-eb68ffe29eaa`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:05,665::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:50:05,679::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:05,679::task::1191::Storage.TaskManager.Task::(prepare) Task=`cac0b550-9b78-460a-8a59-eb68ffe29eaa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:05,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`cac0b550-9b78-460a-8a59-eb68ffe29eaa`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:05,679::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:05,680::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:05,680::task::993::Storage.TaskManager.Task::(_decref) Task=`cac0b550-9b78-460a-8a59-eb68ffe29eaa`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:50:05,690::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:50:05,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`09347932-f3b9-4a33-9dab-029b38cd0aaf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:50:05,691::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:50:05,692::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2c114e36-9687-4805-97bf-e1ea8cce1f0c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:50:05,692::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:50:05,693::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:50:05,693::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2c114e36-9687-4805-97bf-e1ea8cce1f0c`::Granted request >Thread-34::DEBUG::2014-08-11 14:50:05,694::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`09347932-f3b9-4a33-9dab-029b38cd0aaf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:50:05,694::task::993::Storage.TaskManager.Task::(_decref) Task=`09347932-f3b9-4a33-9dab-029b38cd0aaf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:50:05,714::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860247879680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:05,714::task::1191::Storage.TaskManager.Task::(prepare) Task=`09347932-f3b9-4a33-9dab-029b38cd0aaf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860247879680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:05,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`09347932-f3b9-4a33-9dab-029b38cd0aaf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:50:05,715::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:50:05,715::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:50:05,715::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:50:05,716::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:50:05,716::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:50:05,716::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:50:05,716::task::993::Storage.TaskManager.Task::(_decref) Task=`09347932-f3b9-4a33-9dab-029b38cd0aaf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:50:07,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:07,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013196 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:09,034::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:09,035::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:09,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:09,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:09,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133641 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:11,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:11,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131768 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:13,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:13,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132431 s, 77.3 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:50:13,416::task::595::Storage.TaskManager.Task::(_updateState) Task=`408f7c3f-934e-49fc-91bd-c93946a67078`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:50:13,416::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:50:13,417::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:13,418::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:13,418::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:13,418::__init__::232::IOProcess::(_processLogs) (5805) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:50:13,419::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|(5805) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:13,420::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-55::INFO::2014-08-11 14:50:13,421::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:50:13,421::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-55::DEBUG::2014-08-11 14:50:13,422::task::1191::Storage.TaskManager.Task::(prepare) Task=`408f7c3f-934e-49fc-91bd-c93946a67078`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:50:13,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`408f7c3f-934e-49fc-91bd-c93946a67078`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:50:13,423::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:50:13,423::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:50:13,424::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:50:13,425::__init__::232::IOProcess::(_processLogs) (5806) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:50:13,425::task::993::Storage.TaskManager.Task::(_decref) Task=`408f7c3f-934e-49fc-91bd-c93946a67078`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:50:13,426::__init__::232::IOProcess::(_processLogs) (5806) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:14,034::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:14,034::__init__::232::IOProcess::(_processLogs) (5791) Got request for method 'statvfs'DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:14,035::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:14,035::__init__::232::IOProcess::(_processLogs) (5807) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:14,036::__init__::232::IOProcess::(_processLogs) (5807) Queuing response >Thread-16::DEBUG::2014-08-11 14:50:14,036::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:50:14,093::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420335 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:14,094::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:14,095::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:14,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:14,095::__init__::232::IOProcess::(_processLogs) (5808) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:14,096::__init__::232::IOProcess::(_processLogs) DEBUG|(5808) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:14,099::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:14,101::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:14,103::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:14,104::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:14,105::__init__::232::IOProcess::(_processLogs) (5809) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:14,108::__init__::232::IOProcess::(_processLogs) DEBUG|(5809) Queuing response >Thread-103::DEBUG::2014-08-11 14:50:14,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd7f6238-ea5c-40d8-8ed6-2e5dbbe01744`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:14,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::INFO::2014-08-11 14:50:14,112::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-14::DEBUG::2014-08-11 14:50:14,113::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:14,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:14,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-103::INFO::2014-08-11 14:50:14,117::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000596751', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 14:50:14,119::__init__::232::IOProcess::(_processLogs) (5810) Got request for method 'access' >Thread-103::DEBUG::2014-08-11 14:50:14,120::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd7f6238-ea5c-40d8-8ed6-2e5dbbe01744`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000596751', 'lastCheck': '10.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:14,120::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:14,121::__init__::232::IOProcess::(_processLogs) (5810) Queuing response >Thread-103::DEBUG::2014-08-11 14:50:14,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd7f6238-ea5c-40d8-8ed6-2e5dbbe01744`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:14,123::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:50:14,124::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-103::DEBUG::2014-08-11 14:50:14,124::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:50:14,126::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-103::DEBUG::2014-08-11 14:50:14,127::task::993::Storage.TaskManager.Task::(_decref) Task=`dd7f6238-ea5c-40d8-8ed6-2e5dbbe01744`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:50:14,128::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:14,130::__init__::232::IOProcess::(_processLogs) (5811) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:14,132::__init__::232::IOProcess::(_processLogs) (5811) Queuing response >Thread-103::DEBUG::2014-08-11 14:50:14,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:15,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:15,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013221 s, 77.5 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:50:15,729::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:50:15,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d6a6021-0781-46d9-9701-a0d7bf701077`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:15,730::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:50:15,743::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:15,744::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d6a6021-0781-46d9-9701-a0d7bf701077`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:15,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d6a6021-0781-46d9-9701-a0d7bf701077`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:15,744::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:15,744::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:15,744::task::993::Storage.TaskManager.Task::(_decref) Task=`4d6a6021-0781-46d9-9701-a0d7bf701077`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:50:15,754::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:50:15,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`d51a06a9-0eef-4f57-b30f-53ac7d8df325`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:50:15,755::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:50:15,756::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b105b697-1bdb-4e04-bc76-1c77d9de11c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:50:15,756::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:50:15,757::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:50:15,757::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b105b697-1bdb-4e04-bc76-1c77d9de11c4`::Granted request >Thread-34::DEBUG::2014-08-11 14:50:15,758::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d51a06a9-0eef-4f57-b30f-53ac7d8df325`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:50:15,758::task::993::Storage.TaskManager.Task::(_decref) Task=`d51a06a9-0eef-4f57-b30f-53ac7d8df325`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:50:15,778::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860245061632', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:15,778::task::1191::Storage.TaskManager.Task::(prepare) Task=`d51a06a9-0eef-4f57-b30f-53ac7d8df325`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860245061632', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:15,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`d51a06a9-0eef-4f57-b30f-53ac7d8df325`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:50:15,779::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:50:15,779::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:50:15,779::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:50:15,780::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:50:15,780::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:50:15,780::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:50:15,780::task::993::Storage.TaskManager.Task::(_decref) Task=`d51a06a9-0eef-4f57-b30f-53ac7d8df325`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:50:17,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:17,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131213 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:19,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:19,134::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:19,175::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:19,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:19,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158229 s, 64.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:21,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:21,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132843 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:23,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:23,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132836 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:24,128::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:24,129::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:24,129::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:24,130::__init__::232::IOProcess::(_processLogs) (5812) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:24,130::__init__::232::IOProcess::(_processLogs) DEBUG|(5812) Queuing response >Thread-16::DEBUG::2014-08-11 14:50:24,131::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:50:24,149::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425229 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:24,150::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:24,175::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:24,176::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:24,176::__init__::232::IOProcess::(_processLogs) (5813) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:24,177::__init__::232::IOProcess::(_processLogs) (5813) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:24,177::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:24,178::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:24,178::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:24,179::__init__::232::IOProcess::(_processLogs) (5814) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:24,179::__init__::232::IOProcess::(_processLogs) (5814) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:24,179::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:24,179::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:24,180::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:24,180::__init__::232::IOProcess::(_processLogs) (5815) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:24,182::__init__::232::IOProcess::(_processLogs) DEBUG|(5815) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:24,183::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:24,185::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:50:24,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:24,188::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:24,189::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:24,191::__init__::232::IOProcess::(_processLogs) (5816) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:24,193::__init__::232::IOProcess::(_processLogs) (5816) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:24,194::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:25,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:25,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132271 s, 77.4 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:50:25,793::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:50:25,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`9fc1a364-9632-458f-9376-c8750a052918`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:25,795::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:50:25,807::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:25,808::task::1191::Storage.TaskManager.Task::(prepare) Task=`9fc1a364-9632-458f-9376-c8750a052918`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:25,808::task::595::Storage.TaskManager.Task::(_updateState) Task=`9fc1a364-9632-458f-9376-c8750a052918`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:25,808::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:25,808::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:25,808::task::993::Storage.TaskManager.Task::(_decref) Task=`9fc1a364-9632-458f-9376-c8750a052918`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:50:25,820::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:50:25,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2693593-dadb-44e4-8b19-bdc7e649b0de`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:50:25,821::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:50:25,822::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c689d95f-fba8-4a72-9672-6eb8075ec723`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:50:25,822::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:50:25,822::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:50:25,823::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c689d95f-fba8-4a72-9672-6eb8075ec723`::Granted request >Thread-34::DEBUG::2014-08-11 14:50:25,823::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b2693593-dadb-44e4-8b19-bdc7e649b0de`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:50:25,823::task::993::Storage.TaskManager.Task::(_decref) Task=`b2693593-dadb-44e4-8b19-bdc7e649b0de`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:50:25,851::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860241719296', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:25,851::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2693593-dadb-44e4-8b19-bdc7e649b0de`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860241719296', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:25,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2693593-dadb-44e4-8b19-bdc7e649b0de`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:50:25,851::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:50:25,852::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:50:25,852::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:50:25,852::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:50:25,853::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:50:25,853::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:50:25,853::task::993::Storage.TaskManager.Task::(_decref) Task=`b2693593-dadb-44e4-8b19-bdc7e649b0de`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:50:27,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:27,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131975 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:29,202::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:29,203::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:50:29,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`53af9438-88b0-409a-a9ea-b15a53312b6a`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:29,276::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:50:29,278::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425229', 'lastCheck': '5.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:29,279::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:50:29,280::task::1191::Storage.TaskManager.Task::(prepare) Task=`53af9438-88b0-409a-a9ea-b15a53312b6a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425229', 'lastCheck': '5.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:50:29,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`53af9438-88b0-409a-a9ea-b15a53312b6a`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:29,282::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:29,283::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:29,283::task::993::Storage.TaskManager.Task::(_decref) Task=`53af9438-88b0-409a-a9ea-b15a53312b6a`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:50:29,302::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:29,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:29,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132282 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:31,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:31,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133583 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:33,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:33,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131361 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:34,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:34,194::__init__::232::IOProcess::(_processLogs) (5817) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:50:34,195::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:50:34,214::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000431528 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:34,215::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:34,275::__init__::232::IOProcess::(_processLogs) (5818) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:34,277::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:34,277::__init__::232::IOProcess::(_processLogs) (5819) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:34,278::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:34,278::__init__::232::IOProcess::(_processLogs) (5820) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:34,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:34,279::__init__::232::IOProcess::(_processLogs) (5821) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:34,289::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:34,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:34,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:35,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:35,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132773 s, 77.1 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:50:35,867::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-103::DEBUG::2014-08-11 14:50:35,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5d50207-7037-4887-95d6-e6dcf16ffc62`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:35,868::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:50:35,881::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:35,882::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5d50207-7037-4887-95d6-e6dcf16ffc62`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:35,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5d50207-7037-4887-95d6-e6dcf16ffc62`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:35,882::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:35,882::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:35,882::task::993::Storage.TaskManager.Task::(_decref) Task=`b5d50207-7037-4887-95d6-e6dcf16ffc62`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:50:35,893::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 14:50:35,893::task::595::Storage.TaskManager.Task::(_updateState) Task=`7bc55656-bb2d-4910-8bfc-c991342918ae`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:50:35,894::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:50:35,894::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c26fbc8d-43e1-4209-b7b1-7eb30773dcd3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:50:35,895::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:50:35,895::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:50:35,895::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c26fbc8d-43e1-4209-b7b1-7eb30773dcd3`::Granted request >Thread-34::DEBUG::2014-08-11 14:50:35,896::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7bc55656-bb2d-4910-8bfc-c991342918ae`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:50:35,896::task::993::Storage.TaskManager.Task::(_decref) Task=`7bc55656-bb2d-4910-8bfc-c991342918ae`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:50:35,916::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860238049280', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:35,916::task::1191::Storage.TaskManager.Task::(prepare) Task=`7bc55656-bb2d-4910-8bfc-c991342918ae`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860238049280', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:35,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`7bc55656-bb2d-4910-8bfc-c991342918ae`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:50:35,917::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:50:35,917::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:50:35,917::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:50:35,918::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:50:35,918::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:50:35,918::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:50:35,918::task::993::Storage.TaskManager.Task::(_decref) Task=`7bc55656-bb2d-4910-8bfc-c991342918ae`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:50:37,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:37,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132516 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:39,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:39,296::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:39,376::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:39,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:39,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013259 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:41,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:41,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131659 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:44,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:44,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132573 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:44,281::__init__::232::IOProcess::(_processLogs) (5822) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:50:44,282::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:50:44,301::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042512 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:44,303::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:44,375::__init__::232::IOProcess::(_processLogs) (5823) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:44,377::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:44,377::__init__::232::IOProcess::(_processLogs) (5824) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:44,378::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:44,378::__init__::232::IOProcess::(_processLogs) (5825) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:44,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:44,379::__init__::232::IOProcess::(_processLogs) (5826) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:44,380::__init__::232::IOProcess::(_processLogs) (5826) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:44,383::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:44,385::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:44,385::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:50:44,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3a6cfb0-3f37-4b1b-a068-2175907c2159`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:44,393::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:50:44,393::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042512', 'lastCheck': '0.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:50:44,393::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3a6cfb0-3f37-4b1b-a068-2175907c2159`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042512', 'lastCheck': '0.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:50:44,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3a6cfb0-3f37-4b1b-a068-2175907c2159`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:44,394::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:44,394::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:44,394::task::993::Storage.TaskManager.Task::(_decref) Task=`f3a6cfb0-3f37-4b1b-a068-2175907c2159`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:50:44,414::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:50:45,929::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:50:45,930::task::595::Storage.TaskManager.Task::(_updateState) Task=`922d6fe0-3e98-4f1a-9e52-345c70d3b8c6`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:45,930::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:50:45,943::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:45,943::task::1191::Storage.TaskManager.Task::(prepare) Task=`922d6fe0-3e98-4f1a-9e52-345c70d3b8c6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:45,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`922d6fe0-3e98-4f1a-9e52-345c70d3b8c6`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:45,944::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:45,944::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:45,944::task::993::Storage.TaskManager.Task::(_decref) Task=`922d6fe0-3e98-4f1a-9e52-345c70d3b8c6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:50:45,954::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:50:45,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`e55b6b0e-42a7-4ca6-97ac-f400137c187d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:50:45,955::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:50:45,956::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e216c682-cf6c-4348-b415-f58d8823f8ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:50:45,956::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:50:45,956::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:50:45,957::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e216c682-cf6c-4348-b415-f58d8823f8ed`::Granted request >Thread-34::DEBUG::2014-08-11 14:50:45,957::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e55b6b0e-42a7-4ca6-97ac-f400137c187d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:50:45,957::task::993::Storage.TaskManager.Task::(_decref) Task=`e55b6b0e-42a7-4ca6-97ac-f400137c187d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:50:45,977::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860235755520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:45,977::task::1191::Storage.TaskManager.Task::(prepare) Task=`e55b6b0e-42a7-4ca6-97ac-f400137c187d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860235755520', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:45,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`e55b6b0e-42a7-4ca6-97ac-f400137c187d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:50:45,978::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:50:45,978::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:50:45,978::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:50:45,979::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:50:45,979::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:50:45,979::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:50:45,979::task::993::Storage.TaskManager.Task::(_decref) Task=`e55b6b0e-42a7-4ca6-97ac-f400137c187d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:50:46,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:46,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134435 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:48,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:48,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013256 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:49,391::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:49,393::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:49,476::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:50:50,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:50,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143361 s, 71.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:52,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:52,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131741 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:54,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:54,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176374 s, 58.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:54,382::__init__::232::IOProcess::(_processLogs) (5827) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:50:54,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:50:54,402::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412247 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:50:54,403::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:54,475::__init__::232::IOProcess::(_processLogs) (5828) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:50:54,477::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:54,477::__init__::232::IOProcess::(_processLogs) (5829) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:54,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:54,478::__init__::232::IOProcess::(_processLogs) (5830) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:50:54,479::__init__::232::IOProcess::(_processLogs) (5830) Queuing response >Thread-14::DEBUG::2014-08-11 14:50:54,482::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:54,483::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:54,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:50:54,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:50:54,487::__init__::232::IOProcess::(_processLogs) (5831) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:54,488::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:50:54,489::__init__::232::IOProcess::(_processLogs) (5831) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:54,491::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:50:55,991::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:50:55,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`698fd59c-c3e4-49e5-95d7-4e0b42d13067`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:55,993::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:50:56,008::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:56,009::task::1191::Storage.TaskManager.Task::(prepare) Task=`698fd59c-c3e4-49e5-95d7-4e0b42d13067`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:50:56,009::task::595::Storage.TaskManager.Task::(_updateState) Task=`698fd59c-c3e4-49e5-95d7-4e0b42d13067`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:56,009::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:56,009::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:56,009::task::993::Storage.TaskManager.Task::(_decref) Task=`698fd59c-c3e4-49e5-95d7-4e0b42d13067`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:50:56,019::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:50:56,020::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8f3feca-48f1-494c-afd8-a543915c1038`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:50:56,020::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:50:56,021::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`141e372b-6e79-4774-9215-4597e6e03e8d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:50:56,021::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:50:56,022::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:50:56,022::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`141e372b-6e79-4774-9215-4597e6e03e8d`::Granted request >Thread-34::DEBUG::2014-08-11 14:50:56,022::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e8f3feca-48f1-494c-afd8-a543915c1038`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:50:56,023::task::993::Storage.TaskManager.Task::(_decref) Task=`e8f3feca-48f1-494c-afd8-a543915c1038`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:50:56,042::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860221403136', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:56,043::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8f3feca-48f1-494c-afd8-a543915c1038`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860221403136', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:50:56,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8f3feca-48f1-494c-afd8-a543915c1038`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:50:56,043::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:50:56,043::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:50:56,044::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:50:56,044::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:50:56,044::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:50:56,045::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:50:56,045::task::993::Storage.TaskManager.Task::(_decref) Task=`e8f3feca-48f1-494c-afd8-a543915c1038`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:50:56,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:56,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135172 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:50:58,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:50:58,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133384 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:59,495::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:59,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:50:59,499::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:50:59,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`15910a5b-fef3-4397-a257-55a2784ae6e3`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:50:59,513::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:50:59,513::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412247', 'lastCheck': '5.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:50:59,514::task::1191::Storage.TaskManager.Task::(prepare) Task=`15910a5b-fef3-4397-a257-55a2784ae6e3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412247', 'lastCheck': '5.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:50:59,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`15910a5b-fef3-4397-a257-55a2784ae6e3`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:50:59,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:50:59,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:50:59,514::task::993::Storage.TaskManager.Task::(_decref) Task=`15910a5b-fef3-4397-a257-55a2784ae6e3`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:50:59,535::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:00,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:00,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131471 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:02,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:02,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0299682 s, 34.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:04,484::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:04,484::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:04,485::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:04,485::__init__::232::IOProcess::(_processLogs) (5832) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:04,485::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|(5832) Queuing response >Thread-16::DEBUG::2014-08-11 14:51:04,486::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:51:04,507::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418389 s, 1.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:04,508::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:51:04,510::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:04,580::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:04,582::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:04,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:51:04,584::__init__::232::IOProcess::(_processLogs) (5833) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:04,585::__init__::232::IOProcess::(_processLogs) (5833) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:04,588::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:04,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:04,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:04,589::__init__::232::IOProcess::(_processLogs) (5834) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:04,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:51:04,590::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|(5834) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:04,592::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:04,592::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:04,593::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:04,593::__init__::232::IOProcess::(_processLogs) (5835) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:04,594::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|(5835) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:04,594::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:04,595::__init__::232::IOProcess::(_processLogs) (5830) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:04,595::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:04,596::__init__::232::IOProcess::(_processLogs) (5836) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:04,597::__init__::232::IOProcess::(_processLogs) (5836) Queuing response >Dummy-33::DEBUG::2014-08-11 14:51:04,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:04,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132041 s, 77.6 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:51:06,057::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:51:06,058::task::595::Storage.TaskManager.Task::(_updateState) Task=`34e0c39b-99dd-4a3a-b722-e7b44e0c7f06`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:06,058::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:51:06,072::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:06,072::task::1191::Storage.TaskManager.Task::(prepare) Task=`34e0c39b-99dd-4a3a-b722-e7b44e0c7f06`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:06,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`34e0c39b-99dd-4a3a-b722-e7b44e0c7f06`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:06,072::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:06,073::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:51:06,073::task::993::Storage.TaskManager.Task::(_decref) Task=`34e0c39b-99dd-4a3a-b722-e7b44e0c7f06`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:51:06,083::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:51:06,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebe2e21a-cc96-4ece-a7ff-e3fca655d866`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:51:06,084::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:51:06,085::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`be0a0dfb-75bf-4f81-a060-9c6dc367af05`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:51:06,085::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:51:06,086::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:51:06,086::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`be0a0dfb-75bf-4f81-a060-9c6dc367af05`::Granted request >Thread-34::DEBUG::2014-08-11 14:51:06,087::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ebe2e21a-cc96-4ece-a7ff-e3fca655d866`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:51:06,087::task::993::Storage.TaskManager.Task::(_decref) Task=`ebe2e21a-cc96-4ece-a7ff-e3fca655d866`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:51:06,107::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860214521856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:06,107::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebe2e21a-cc96-4ece-a7ff-e3fca655d866`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860214521856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:06,107::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebe2e21a-cc96-4ece-a7ff-e3fca655d866`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:51:06,108::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:51:06,108::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:51:06,108::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:51:06,108::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:51:06,109::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:51:06,109::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:51:06,109::task::993::Storage.TaskManager.Task::(_decref) Task=`ebe2e21a-cc96-4ece-a7ff-e3fca655d866`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:51:06,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:06,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131973 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:08,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:08,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134237 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:09,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:09,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:09,678::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:10,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:11,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131864 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:13,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:13,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131454 s, 77.9 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:51:13,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`05b8bf4e-0e5c-4c7d-82dd-888b1ce6710b`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:51:13,537::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:51:13,538::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:13,538::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:13,539::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:13,539::__init__::232::IOProcess::(_processLogs) (5837) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:51:13,539::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5837) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:13,540::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:13,541::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:13,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:13,541::__init__::232::IOProcess::(_processLogs) (5838) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:51:13,541::__init__::232::IOProcess::(_processLogs) (5838) Queuing response >Thread-55::INFO::2014-08-11 14:51:13,542::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:51:13,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`05b8bf4e-0e5c-4c7d-82dd-888b1ce6710b`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:51:13,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`05b8bf4e-0e5c-4c7d-82dd-888b1ce6710b`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:51:13,543::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:51:13,543::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:51:13,543::task::993::Storage.TaskManager.Task::(_decref) Task=`05b8bf4e-0e5c-4c7d-82dd-888b1ce6710b`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:51:14,597::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:14,598::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:14,598::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:14,598::__init__::232::IOProcess::(_processLogs) (5839) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:51:14,599::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:51:14,618::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041438 s, 1.1 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:51:14,631::task::595::Storage.TaskManager.Task::(_updateState) Task=`33dceb88-89ce-4a4b-bb0c-0161dc51bf1d`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:14,680::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-14::DEBUG::2014-08-11 14:51:14,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-103::INFO::2014-08-11 14:51:14,682::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418389', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 14:51:14,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-103::DEBUG::2014-08-11 14:51:14,683::task::1191::Storage.TaskManager.Task::(prepare) Task=`33dceb88-89ce-4a4b-bb0c-0161dc51bf1d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418389', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 14:51:14,683::__init__::232::IOProcess::(_processLogs) (5840) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:14,686::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:51:14,686::task::595::Storage.TaskManager.Task::(_updateState) Task=`33dceb88-89ce-4a4b-bb0c-0161dc51bf1d`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:14,691::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:51:14,692::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-103::DEBUG::2014-08-11 14:51:14,693::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 14:51:14,694::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:14,696::__init__::232::IOProcess::(_processLogs) (5841) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:14,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:51:14,697::task::993::Storage.TaskManager.Task::(_decref) Task=`33dceb88-89ce-4a4b-bb0c-0161dc51bf1d`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 14:51:14,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:14,705::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:14,706::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:51:14,707::__init__::232::IOProcess::(_processLogs) (5842) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:14,709::__init__::232::IOProcess::(_processLogs) (5842) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:14,714::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:14,715::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:14,717::__init__::232::IOProcess::(_processLogs) (5843) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:14,718::__init__::232::IOProcess::(_processLogs) (5843) Queuing response >Thread-103::DEBUG::2014-08-11 14:51:14,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:15,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:15,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132839 s, 77.1 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:51:16,122::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:51:16,123::task::595::Storage.TaskManager.Task::(_updateState) Task=`e75a887b-8a89-4947-9cb4-d24b34c71ba4`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:16,123::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:51:16,136::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:16,136::task::1191::Storage.TaskManager.Task::(prepare) Task=`e75a887b-8a89-4947-9cb4-d24b34c71ba4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:16,137::task::595::Storage.TaskManager.Task::(_updateState) Task=`e75a887b-8a89-4947-9cb4-d24b34c71ba4`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:16,137::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:16,137::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:51:16,137::task::993::Storage.TaskManager.Task::(_decref) Task=`e75a887b-8a89-4947-9cb4-d24b34c71ba4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:51:16,152::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:51:16,153::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5a74c7a-db7e-4e50-b9b4-41db171c54da`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:51:16,153::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:51:16,154::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`18bddba9-c801-474d-a701-523fdfdcca8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:51:16,154::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:51:16,155::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:51:16,155::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`18bddba9-c801-474d-a701-523fdfdcca8a`::Granted request >Thread-34::DEBUG::2014-08-11 14:51:16,156::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5a74c7a-db7e-4e50-b9b4-41db171c54da`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:51:16,156::task::993::Storage.TaskManager.Task::(_decref) Task=`d5a74c7a-db7e-4e50-b9b4-41db171c54da`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:51:16,176::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860209410048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:16,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5a74c7a-db7e-4e50-b9b4-41db171c54da`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860209410048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:16,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5a74c7a-db7e-4e50-b9b4-41db171c54da`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:51:16,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:51:16,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:51:16,177::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:51:16,178::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:51:16,178::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:51:16,178::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:51:16,178::task::993::Storage.TaskManager.Task::(_decref) Task=`d5a74c7a-db7e-4e50-b9b4-41db171c54da`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:51:17,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:17,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132102 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:19,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:19,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132844 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:19,719::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:19,720::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:19,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:21,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:21,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132467 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:23,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:23,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135973 s, 75.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:24,716::__init__::232::IOProcess::(_processLogs) (5844) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:51:24,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:51:24,736::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421426 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:24,737::__init__::232::IOProcess::(_processLogs) (5845) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:24,780::__init__::232::IOProcess::(_processLogs) (5846) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:24,781::__init__::232::IOProcess::(_processLogs) (5847) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:24,782::__init__::232::IOProcess::(_processLogs) (5848) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:24,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:24,787::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:24,788::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:25,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:25,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132306 s, 77.4 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:51:26,190::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:51:26,191::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a90c000-7f9f-435f-a6ef-62805d0324c4`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:26,191::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:51:26,204::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:26,204::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a90c000-7f9f-435f-a6ef-62805d0324c4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:26,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a90c000-7f9f-435f-a6ef-62805d0324c4`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:26,205::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:26,205::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:51:26,205::task::993::Storage.TaskManager.Task::(_decref) Task=`2a90c000-7f9f-435f-a6ef-62805d0324c4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:51:26,216::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:51:26,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`d44de790-1e03-4e43-a422-afb8c59454f8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:51:26,217::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:51:26,217::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a2dd13c-f837-443f-9158-264bee3fda56`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:51:26,218::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:51:26,218::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:51:26,218::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a2dd13c-f837-443f-9158-264bee3fda56`::Granted request >Thread-34::DEBUG::2014-08-11 14:51:26,219::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d44de790-1e03-4e43-a422-afb8c59454f8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:51:26,219::task::993::Storage.TaskManager.Task::(_decref) Task=`d44de790-1e03-4e43-a422-afb8c59454f8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:51:26,239::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860177887232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:26,239::task::1191::Storage.TaskManager.Task::(prepare) Task=`d44de790-1e03-4e43-a422-afb8c59454f8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860177887232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:26,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`d44de790-1e03-4e43-a422-afb8c59454f8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:51:26,240::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:51:26,240::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:51:26,240::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:51:26,241::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:51:26,241::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:51:26,241::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:51:26,242::task::993::Storage.TaskManager.Task::(_decref) Task=`d44de790-1e03-4e43-a422-afb8c59454f8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:51:27,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:27,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132171 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:29,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:29,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136567 s, 75.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:29,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:51:29,828::task::595::Storage.TaskManager.Task::(_updateState) Task=`426c8f32-d99d-475c-9ceb-6d2729294150`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:29,881::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:51:29,883::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421426', 'lastCheck': '5.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:29,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:51:29,885::task::1191::Storage.TaskManager.Task::(prepare) Task=`426c8f32-d99d-475c-9ceb-6d2729294150`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421426', 'lastCheck': '5.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:51:29,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`426c8f32-d99d-475c-9ceb-6d2729294150`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:29,887::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:29,889::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:29,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:51:29,891::task::993::Storage.TaskManager.Task::(_decref) Task=`426c8f32-d99d-475c-9ceb-6d2729294150`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:51:29,911::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:31,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:31,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133128 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:33,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:33,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133702 s, 76.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:34,784::__init__::232::IOProcess::(_processLogs) (5849) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:51:34,785::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:51:34,803::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419123 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:34,805::__init__::232::IOProcess::(_processLogs) (5850) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:34,881::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:34,881::__init__::232::IOProcess::(_processLogs) (5851) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:34,882::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:34,882::__init__::232::IOProcess::(_processLogs) (5852) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:34,883::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:34,883::__init__::232::IOProcess::(_processLogs) (5853) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:34,884::__init__::232::IOProcess::(_processLogs) (5853) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:34,896::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:34,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:34,898::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:35,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:35,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131897 s, 77.6 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:51:36,254::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:51:36,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f00cafd-c5f8-4c08-986d-b3e9ca90e25a`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:36,255::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:51:36,268::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:36,268::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f00cafd-c5f8-4c08-986d-b3e9ca90e25a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:36,268::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f00cafd-c5f8-4c08-986d-b3e9ca90e25a`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:36,269::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:36,269::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:51:36,269::task::993::Storage.TaskManager.Task::(_decref) Task=`3f00cafd-c5f8-4c08-986d-b3e9ca90e25a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:51:36,279::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:51:36,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`957fb0d4-2d4b-4e5f-bc82-49177fb22cb2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:51:36,280::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:51:36,281::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5332b6a4-0e41-48c8-ae09-df2233b70674`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:51:36,281::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:51:36,282::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:51:36,282::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5332b6a4-0e41-48c8-ae09-df2233b70674`::Granted request >Thread-34::DEBUG::2014-08-11 14:51:36,282::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`957fb0d4-2d4b-4e5f-bc82-49177fb22cb2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:51:36,283::task::993::Storage.TaskManager.Task::(_decref) Task=`957fb0d4-2d4b-4e5f-bc82-49177fb22cb2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:51:36,302::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860001398784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:36,303::task::1191::Storage.TaskManager.Task::(prepare) Task=`957fb0d4-2d4b-4e5f-bc82-49177fb22cb2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1860001398784', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:36,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`957fb0d4-2d4b-4e5f-bc82-49177fb22cb2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:51:36,303::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:51:36,303::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:51:36,304::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:51:36,304::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:51:36,304::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:51:36,305::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:51:36,305::task::993::Storage.TaskManager.Task::(_decref) Task=`957fb0d4-2d4b-4e5f-bc82-49177fb22cb2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:51:37,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:37,721::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133247 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:39,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:39,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131677 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:39,904::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:39,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:39,980::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:41,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:41,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132467 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:43,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:43,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0205448 s, 49.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:44,885::__init__::232::IOProcess::(_processLogs) (5853) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:44,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:51:44,887::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:51:44,888::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:44,906::__init__::232::IOProcess::(_processLogs) (5854) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:44,907::__init__::232::IOProcess::(_processLogs) (5854) Queuing response >Thread-16::DEBUG::2014-08-11 14:51:44,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000630703 s, 721 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:44,981::__init__::232::IOProcess::(_processLogs) (5855) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:44,982::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:44,982::__init__::232::IOProcess::(_processLogs) (5856) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:44,983::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:44,984::__init__::232::IOProcess::(_processLogs) (5857) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:44,984::__init__::232::IOProcess::(_processLogs) (5857) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:44,985::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:44,986::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:44,986::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:44,989::__init__::232::IOProcess::(_processLogs) (5858) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:44,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:51:44,990::__init__::232::IOProcess::(_processLogs) (5858) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:44,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:44,993::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:51:45,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d44b70d-d9d5-405c-9773-4001fa32cb8e`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:45,000::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:51:45,000::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000630703', 'lastCheck': '0.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:51:45,001::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d44b70d-d9d5-405c-9773-4001fa32cb8e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000630703', 'lastCheck': '0.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:51:45,001::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d44b70d-d9d5-405c-9773-4001fa32cb8e`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:45,001::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:45,001::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:51:45,002::task::993::Storage.TaskManager.Task::(_decref) Task=`9d44b70d-d9d5-405c-9773-4001fa32cb8e`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:51:45,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:45,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:45,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136826 s, 74.8 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:51:46,318::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:51:46,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`2903b82c-977b-44aa-8596-06c8cd1ff67d`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:46,319::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:51:46,332::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:46,332::task::1191::Storage.TaskManager.Task::(prepare) Task=`2903b82c-977b-44aa-8596-06c8cd1ff67d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:46,333::task::595::Storage.TaskManager.Task::(_updateState) Task=`2903b82c-977b-44aa-8596-06c8cd1ff67d`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:46,333::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:46,333::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:51:46,333::task::993::Storage.TaskManager.Task::(_decref) Task=`2903b82c-977b-44aa-8596-06c8cd1ff67d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:51:46,343::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:51:46,344::task::595::Storage.TaskManager.Task::(_updateState) Task=`4415af6b-045f-4822-a288-f3db42423a7a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:51:46,344::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:51:46,345::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`63fbbd54-7494-4e1e-9b8e-d6efbdec12eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:51:46,345::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:51:46,345::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:51:46,345::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`63fbbd54-7494-4e1e-9b8e-d6efbdec12eb`::Granted request >Thread-34::DEBUG::2014-08-11 14:51:46,346::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4415af6b-045f-4822-a288-f3db42423a7a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:51:46,346::task::993::Storage.TaskManager.Task::(_decref) Task=`4415af6b-045f-4822-a288-f3db42423a7a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:51:46,366::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859605168128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:46,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`4415af6b-045f-4822-a288-f3db42423a7a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859605168128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:46,367::task::595::Storage.TaskManager.Task::(_updateState) Task=`4415af6b-045f-4822-a288-f3db42423a7a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:51:46,367::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:51:46,367::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:51:46,368::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:51:46,368::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:51:46,368::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:51:46,368::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:51:46,369::task::993::Storage.TaskManager.Task::(_decref) Task=`4415af6b-045f-4822-a288-f3db42423a7a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:51:47,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:48,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131308 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:50,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:50,002::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:50,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:50,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:50,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136501 s, 75.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:52,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:52,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131559 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:51:54,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:54,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131441 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:54,989::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:54,990::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:54,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:51:54,991::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:51:54,992::__init__::232::IOProcess::(_processLogs) (5859) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:54,993::__init__::232::IOProcess::(_processLogs) (5859) Queuing response >Thread-16::DEBUG::2014-08-11 14:51:55,011::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000466377 s, 976 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:51:55,080::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:55,081::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:55,081::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:55,081::__init__::232::IOProcess::(_processLogs) (5860) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:51:55,082::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5860) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:55,083::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:55,084::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:55,084::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:55,084::__init__::232::IOProcess::(_processLogs) (5861) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:55,085::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5861) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:55,085::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:55,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:55,086::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:55,086::__init__::232::IOProcess::(_processLogs) (5862) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:55,086::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5862) Queuing response >Thread-14::DEBUG::2014-08-11 14:51:55,087::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:51:55,088::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:51:55,088::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:51:55,088::__init__::232::IOProcess::(_processLogs) (5863) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:51:55,088::__init__::232::IOProcess::(_processLogs) (5863) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:55,103::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:55,104::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:51:55,105::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:51:56,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:56,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132806 s, 77.1 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:51:56,380::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:51:56,381::task::595::Storage.TaskManager.Task::(_updateState) Task=`2653d2c7-61df-42f4-a8c6-1a0620646d5d`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:51:56,381::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:51:56,394::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:56,394::task::1191::Storage.TaskManager.Task::(prepare) Task=`2653d2c7-61df-42f4-a8c6-1a0620646d5d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:51:56,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`2653d2c7-61df-42f4-a8c6-1a0620646d5d`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:51:56,395::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:51:56,395::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:51:56,395::task::993::Storage.TaskManager.Task::(_decref) Task=`2653d2c7-61df-42f4-a8c6-1a0620646d5d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:51:56,407::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:51:56,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a47b678-879f-42ab-82d1-854d45c33111`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:51:56,408::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:51:56,409::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9ebb480-b263-45fb-9dd1-8db2b82a11d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:51:56,409::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:51:56,410::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:51:56,410::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9ebb480-b263-45fb-9dd1-8db2b82a11d2`::Granted request >Thread-34::DEBUG::2014-08-11 14:51:56,410::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1a47b678-879f-42ab-82d1-854d45c33111`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:51:56,411::task::993::Storage.TaskManager.Task::(_decref) Task=`1a47b678-879f-42ab-82d1-854d45c33111`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:51:56,430::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859605168128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:56,431::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a47b678-879f-42ab-82d1-854d45c33111`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859605168128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:51:56,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a47b678-879f-42ab-82d1-854d45c33111`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:51:56,431::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:51:56,431::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:51:56,432::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:51:56,432::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:51:56,432::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:51:56,433::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:51:56,433::task::993::Storage.TaskManager.Task::(_decref) Task=`1a47b678-879f-42ab-82d1-854d45c33111`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:51:58,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:51:58,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0293361 s, 34.9 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:52:00,106::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c46fc01-e0a8-4eca-afb6-3491064b7083`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:00,106::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:00,112::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::INFO::2014-08-11 14:52:00,181::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000466377', 'lastCheck': '5.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:52:00,182::task::1191::Storage.TaskManager.Task::(prepare) Task=`7c46fc01-e0a8-4eca-afb6-3491064b7083`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000466377', 'lastCheck': '5.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:52:00,183::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c46fc01-e0a8-4eca-afb6-3491064b7083`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:00,185::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:00,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:00,186::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:00,187::task::993::Storage.TaskManager.Task::(_decref) Task=`7c46fc01-e0a8-4eca-afb6-3491064b7083`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:00,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:00,214::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:00,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:00,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133158 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:02,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:02,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244792 s, 41.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:04,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:04,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131992 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:05,090::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:05,091::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:05,091::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:05,091::__init__::232::IOProcess::(_processLogs) (5864) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:05,092::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5864) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:05,092::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:52:05,112::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000788921 s, 577 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:05,113::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:05,114::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:05,114::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:05,114::__init__::232::IOProcess::(_processLogs) (5865) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:05,115::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5865) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:05,116::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:05,116::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:05,117::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:05,117::__init__::232::IOProcess::(_processLogs) (5866) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:05,117::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5866) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:05,118::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:05,118::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:05,118::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:05,119::__init__::232::IOProcess::(_processLogs) (5867) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:05,119::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5867) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:05,120::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:05,120::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:05,120::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:05,120::__init__::232::IOProcess::(_processLogs) (5868) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:05,121::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5868) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:05,202::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:05,207::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:05,213::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:06,445::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-103::DEBUG::2014-08-11 14:52:06,446::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccd9af16-e01f-4de3-9fb4-5d31a8936c16`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:06,446::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:52:06,460::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:06,460::task::1191::Storage.TaskManager.Task::(prepare) Task=`ccd9af16-e01f-4de3-9fb4-5d31a8936c16`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:06,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccd9af16-e01f-4de3-9fb4-5d31a8936c16`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:06,461::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:06,461::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:06,461::task::993::Storage.TaskManager.Task::(_decref) Task=`ccd9af16-e01f-4de3-9fb4-5d31a8936c16`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:52:06,471::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 14:52:06,472::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfab05e9-e1fe-49a6-9f1b-24745fbfd8bd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:52:06,472::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:52:06,473::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b5ad3c4a-0dd9-4811-bebb-99061e50783c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:52:06,473::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:52:06,473::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:52:06,473::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b5ad3c4a-0dd9-4811-bebb-99061e50783c`::Granted request >Thread-34::DEBUG::2014-08-11 14:52:06,474::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bfab05e9-e1fe-49a6-9f1b-24745fbfd8bd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:52:06,474::task::993::Storage.TaskManager.Task::(_decref) Task=`bfab05e9-e1fe-49a6-9f1b-24745fbfd8bd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:52:06,494::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859605168128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:06,494::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfab05e9-e1fe-49a6-9f1b-24745fbfd8bd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859605168128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:06,495::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfab05e9-e1fe-49a6-9f1b-24745fbfd8bd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:52:06,495::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:52:06,495::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:52:06,495::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:52:06,496::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:52:06,496::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:52:06,496::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:52:06,496::task::993::Storage.TaskManager.Task::(_decref) Task=`bfab05e9-e1fe-49a6-9f1b-24745fbfd8bd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:52:06,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:06,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131456 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:08,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:08,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132181 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:10,224::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:10,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:10,280::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:10,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:10,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132295 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:12,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:12,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131582 s, 77.8 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:52:13,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`8519c4ae-1ed6-4989-827a-65b037ea988a`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:52:13,650::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:52:13,651::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:13,651::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:13,652::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:13,652::__init__::232::IOProcess::(_processLogs) (5869) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:52:13,653::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5869) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:13,654::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:13,654::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:13,654::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:13,655::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5870) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:52:13,655::__init__::232::IOProcess::(_processLogs) (5870) Queuing response >Thread-55::INFO::2014-08-11 14:52:13,655::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:52:13,656::task::1191::Storage.TaskManager.Task::(prepare) Task=`8519c4ae-1ed6-4989-827a-65b037ea988a`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:52:13,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`8519c4ae-1ed6-4989-827a-65b037ea988a`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:52:13,656::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:52:13,656::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:52:13,657::task::993::Storage.TaskManager.Task::(_decref) Task=`8519c4ae-1ed6-4989-827a-65b037ea988a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:52:14,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:14,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132746 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:15,123::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:15,123::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:15,124::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:15,124::__init__::232::IOProcess::(_processLogs) (5871) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:15,124::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5871) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:15,125::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:52:15,144::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041714 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:15,145::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:15,145::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:15,146::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:15,146::__init__::232::IOProcess::(_processLogs) (5872) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:15,147::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5872) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:15,148::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:15,148::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:15,149::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:15,149::__init__::232::IOProcess::(_processLogs) (5873) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:15,149::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5873) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:15,150::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:15,151::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:15,151::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:15,151::__init__::232::IOProcess::(_processLogs) (5874) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:15,152::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5874) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:15,153::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:15,153::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:15,153::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:15,153::__init__::232::IOProcess::(_processLogs) (5875) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:15,154::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5875) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:15,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:15,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:15,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`22485b5a-fb76-4f90-a689-6dff051386b1`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:15,381::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:52:15,381::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041714', 'lastCheck': '0.2', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:52:15,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`22485b5a-fb76-4f90-a689-6dff051386b1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041714', 'lastCheck': '0.2', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:52:15,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`22485b5a-fb76-4f90-a689-6dff051386b1`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:15,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:15,383::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:15,384::task::993::Storage.TaskManager.Task::(_decref) Task=`22485b5a-fb76-4f90-a689-6dff051386b1`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:15,387::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:15,405::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:16,508::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:52:16,509::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd35e085-9dea-4c01-b73e-706674dfc28b`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:16,509::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:52:16,522::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:16,522::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd35e085-9dea-4c01-b73e-706674dfc28b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:16,522::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd35e085-9dea-4c01-b73e-706674dfc28b`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:16,522::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:16,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:16,523::task::993::Storage.TaskManager.Task::(_decref) Task=`cd35e085-9dea-4c01-b73e-706674dfc28b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:52:16,534::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:52:16,534::task::595::Storage.TaskManager.Task::(_updateState) Task=`0391d83a-eeac-41f4-bc48-2c37319549a6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:52:16,535::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:52:16,535::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f639ae5e-1d7c-4834-a486-d9c2c1334079`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:52:16,536::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:52:16,536::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:52:16,536::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f639ae5e-1d7c-4834-a486-d9c2c1334079`::Granted request >Thread-34::DEBUG::2014-08-11 14:52:16,537::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0391d83a-eeac-41f4-bc48-2c37319549a6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:52:16,537::task::993::Storage.TaskManager.Task::(_decref) Task=`0391d83a-eeac-41f4-bc48-2c37319549a6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:52:16,557::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859602153472', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:16,557::task::1191::Storage.TaskManager.Task::(prepare) Task=`0391d83a-eeac-41f4-bc48-2c37319549a6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859602153472', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:16,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`0391d83a-eeac-41f4-bc48-2c37319549a6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:52:16,557::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:52:16,558::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:52:16,558::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:52:16,558::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:52:16,559::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:52:16,559::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:52:16,559::task::993::Storage.TaskManager.Task::(_decref) Task=`0391d83a-eeac-41f4-bc48-2c37319549a6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:52:17,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:17,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134898 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:19,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:19,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133225 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:20,395::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:20,396::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:20,480::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:21,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:21,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132274 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:23,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:23,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132614 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:25,156::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:25,156::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:25,156::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:25,157::__init__::232::IOProcess::(_processLogs) (5876) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:25,157::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|(5876) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:25,158::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:52:25,176::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421385 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:25,178::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:25,178::__init__::232::IOProcess::(_processLogs) (5857) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:25,178::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:25,179::__init__::232::IOProcess::(_processLogs) (5877) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:25,180::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:25,180::__init__::232::IOProcess::(_processLogs) (5878) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:25,180::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5878) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:25,181::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:25,182::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:25,182::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:25,182::__init__::232::IOProcess::(_processLogs) (5879) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:25,182::__init__::232::IOProcess::(_processLogs) (5879) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:25,183::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:25,183::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:25,184::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:25,184::__init__::232::IOProcess::(_processLogs) (5880) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:25,184::__init__::232::IOProcess::(_processLogs) (5880) Queuing response >Dummy-33::DEBUG::2014-08-11 14:52:25,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:25,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131924 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:25,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:25,488::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:25,489::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:26,572::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:52:26,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffef6288-59d1-4bac-89eb-a45686aaa8d4`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:26,573::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:52:26,586::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:26,586::task::1191::Storage.TaskManager.Task::(prepare) Task=`ffef6288-59d1-4bac-89eb-a45686aaa8d4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:26,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffef6288-59d1-4bac-89eb-a45686aaa8d4`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:26,587::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:26,587::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:26,587::task::993::Storage.TaskManager.Task::(_decref) Task=`ffef6288-59d1-4bac-89eb-a45686aaa8d4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:52:26,598::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:52:26,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba78d0f1-35ee-414b-8367-8cbc238ff541`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:52:26,599::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:52:26,599::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`95d6d717-676d-46fc-973d-97bf6d19ed50`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:52:26,600::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:52:26,600::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:52:26,600::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`95d6d717-676d-46fc-973d-97bf6d19ed50`::Granted request >Thread-34::DEBUG::2014-08-11 14:52:26,601::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ba78d0f1-35ee-414b-8367-8cbc238ff541`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:52:26,601::task::993::Storage.TaskManager.Task::(_decref) Task=`ba78d0f1-35ee-414b-8367-8cbc238ff541`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:52:26,621::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859599138816', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:26,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba78d0f1-35ee-414b-8367-8cbc238ff541`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859599138816', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:26,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba78d0f1-35ee-414b-8367-8cbc238ff541`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:52:26,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:52:26,622::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:52:26,622::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:52:26,622::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:52:26,623::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:52:26,623::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:52:26,623::task::993::Storage.TaskManager.Task::(_decref) Task=`ba78d0f1-35ee-414b-8367-8cbc238ff541`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:52:27,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:27,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131851 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:29,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:29,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132847 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:30,491::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:30,493::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:30,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`21734a22-3528-4091-bf90-020e64499ef5`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:30,581::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:52:30,583::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421385', 'lastCheck': '5.4', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:52:30,584::task::1191::Storage.TaskManager.Task::(prepare) Task=`21734a22-3528-4091-bf90-020e64499ef5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421385', 'lastCheck': '5.4', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:30,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:30,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`21734a22-3528-4091-bf90-020e64499ef5`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:30,586::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:30,587::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:30,589::task::993::Storage.TaskManager.Task::(_decref) Task=`21734a22-3528-4091-bf90-020e64499ef5`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:52:30,611::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:31,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:31,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135278 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:33,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:33,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132132 s, 77.5 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:52:35,185::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:52:35,187::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:35,188::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:35,188::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,188::__init__::232::IOProcess::(_processLogs) (5881) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:52:35,189::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5881) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:35,199::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:52:35,200::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:35,200::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:35,200::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,200::__init__::232::IOProcess::(_processLogs) (5882) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 14:52:35,201::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5882) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:35,201::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:52:35,202::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:35,202::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:35,203::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,203::__init__::232::IOProcess::(_processLogs) (5883) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:52:35,203::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5883) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:35,204::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:52:35,206::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:35,206::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:35,207::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,207::__init__::232::IOProcess::(_processLogs) (5884) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:35,207::__init__::232::IOProcess::(_processLogs) (5884) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:35,208::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:35,208::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:35,208::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,208::__init__::232::IOProcess::(_processLogs) (5885) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 14:52:35,209::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 14:52:35,209::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5885) Queuing response >Thread-16::INFO::2014-08-11 14:52:35,210::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:52:35,211::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:52:35,211::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:35,212::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:35,212::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,212::__init__::232::IOProcess::(_processLogs) (5886) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:35,213::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5886) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:35,214::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:52:35,233::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000662022 s, 687 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:35,234::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:35,235::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:35,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,235::__init__::232::IOProcess::(_processLogs) (5887) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:35,236::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,237::__init__::232::IOProcess::(_processLogs) (5888) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:35,238::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,238::__init__::232::IOProcess::(_processLogs) (5889) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:35,239::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:35,239::__init__::232::IOProcess::(_processLogs) (5890) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:35,239::__init__::232::IOProcess::(_processLogs) (5890) Queuing response >Dummy-33::DEBUG::2014-08-11 14:52:35,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:35,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0269706 s, 38.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:35,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:35,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:35,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:36,635::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:52:36,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`e35ecaf3-cf4a-4c1c-b24b-c925da52942a`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:36,636::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:52:36,649::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:36,650::task::1191::Storage.TaskManager.Task::(prepare) Task=`e35ecaf3-cf4a-4c1c-b24b-c925da52942a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:36,650::task::595::Storage.TaskManager.Task::(_updateState) Task=`e35ecaf3-cf4a-4c1c-b24b-c925da52942a`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:36,650::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:36,650::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:36,650::task::993::Storage.TaskManager.Task::(_decref) Task=`e35ecaf3-cf4a-4c1c-b24b-c925da52942a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:52:36,660::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:52:36,661::task::595::Storage.TaskManager.Task::(_updateState) Task=`9649535e-2659-413d-b28d-7616e7ddfcb6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:52:36,661::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:52:36,662::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6630807d-0ecd-4948-86b8-9a78d124105d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:52:36,662::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:52:36,663::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:52:36,663::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6630807d-0ecd-4948-86b8-9a78d124105d`::Granted request >Thread-34::DEBUG::2014-08-11 14:52:36,663::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9649535e-2659-413d-b28d-7616e7ddfcb6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:52:36,664::task::993::Storage.TaskManager.Task::(_decref) Task=`9649535e-2659-413d-b28d-7616e7ddfcb6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:52:36,689::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859542712320', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:36,689::task::1191::Storage.TaskManager.Task::(prepare) Task=`9649535e-2659-413d-b28d-7616e7ddfcb6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859542712320', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:36,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`9649535e-2659-413d-b28d-7616e7ddfcb6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:52:36,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:52:36,690::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:52:36,690::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:52:36,691::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:52:36,691::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:52:36,691::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:52:36,691::task::993::Storage.TaskManager.Task::(_decref) Task=`9649535e-2659-413d-b28d-7616e7ddfcb6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:52:37,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:37,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132677 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:39,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:39,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132438 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:40,617::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:40,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:40,680::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:41,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:41,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131436 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:43,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:43,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132336 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:45,241::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:45,242::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:52:45,244::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:52:45,244::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:45,262::__init__::232::IOProcess::(_processLogs) (5891) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:45,263::__init__::232::IOProcess::(_processLogs) (5891) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:45,264::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000558971 s, 814 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:45,265::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:45,265::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:45,266::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:45,266::__init__::232::IOProcess::(_processLogs) (5892) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:45,267::__init__::232::IOProcess::(_processLogs) DEBUG|(5892) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:45,268::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:45,268::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:45,269::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:45,269::__init__::232::IOProcess::(_processLogs) (5893) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:45,270::__init__::232::IOProcess::(_processLogs) DEBUG|(5893) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:45,270::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:45,270::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:45,271::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:45,271::__init__::232::IOProcess::(_processLogs) (5894) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:45,272::__init__::232::IOProcess::(_processLogs) DEBUG|(5894) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:45,272::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:45,273::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:45,273::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:45,273::__init__::232::IOProcess::(_processLogs) (5895) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:45,274::__init__::232::IOProcess::(_processLogs) DEBUG|(5895) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:45,688::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:45,695::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:45,727::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a531ef6-d819-44ad-bdf3-f02bde9a0295`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:45,781::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:52:45,782::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000558971', 'lastCheck': '0.5', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:52:45,782::task::1191::Storage.TaskManager.Task::(prepare) Task=`0a531ef6-d819-44ad-bdf3-f02bde9a0295`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000558971', 'lastCheck': '0.5', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:52:45,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a531ef6-d819-44ad-bdf3-f02bde9a0295`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:45,783::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:45,783::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:45,784::task::993::Storage.TaskManager.Task::(_decref) Task=`0a531ef6-d819-44ad-bdf3-f02bde9a0295`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:45,784::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:52:45,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:45,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:45,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131771 s, 77.7 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:52:46,703::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-103::DEBUG::2014-08-11 14:52:46,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccefa036-7dc1-495a-83e1-fcfd4710c9a3`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:46,704::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:52:46,717::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:46,717::task::1191::Storage.TaskManager.Task::(prepare) Task=`ccefa036-7dc1-495a-83e1-fcfd4710c9a3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:46,718::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccefa036-7dc1-495a-83e1-fcfd4710c9a3`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:46,718::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:46,718::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:46,718::task::993::Storage.TaskManager.Task::(_decref) Task=`ccefa036-7dc1-495a-83e1-fcfd4710c9a3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:52:46,728::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 14:52:46,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`94fea28f-ce6f-4dae-925f-74d053cf9831`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:52:46,729::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:52:46,730::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c94903cf-f97c-4ab9-9573-854a14449560`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:52:46,730::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:52:46,731::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:52:46,731::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c94903cf-f97c-4ab9-9573-854a14449560`::Granted request >Thread-34::DEBUG::2014-08-11 14:52:46,731::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`94fea28f-ce6f-4dae-925f-74d053cf9831`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:52:46,732::task::993::Storage.TaskManager.Task::(_decref) Task=`94fea28f-ce6f-4dae-925f-74d053cf9831`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:52:46,751::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859539632128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:46,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`94fea28f-ce6f-4dae-925f-74d053cf9831`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859539632128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:46,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`94fea28f-ce6f-4dae-925f-74d053cf9831`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:52:46,752::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:52:46,753::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:52:46,753::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:52:46,753::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:52:46,753::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:52:46,754::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:52:46,754::task::993::Storage.TaskManager.Task::(_decref) Task=`94fea28f-ce6f-4dae-925f-74d053cf9831`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:52:47,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:47,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133562 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:49,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:50,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132822 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:50,791::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:50,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:50,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:52,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:52,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132363 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:52:54,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:54,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132097 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:55,276::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:55,276::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:55,277::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:55,277::__init__::232::IOProcess::(_processLogs) (5896) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:55,277::__init__::232::IOProcess::(_processLogs) DEBUG|(5896) Queuing response >Thread-16::DEBUG::2014-08-11 14:52:55,279::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:52:55,298::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426211 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:52:55,299::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:55,300::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:55,300::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:55,300::__init__::232::IOProcess::(_processLogs) (5897) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:52:55,301::__init__::232::IOProcess::(_processLogs) DEBUG|(5897) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:55,302::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:55,303::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:55,303::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:55,304::__init__::232::IOProcess::(_processLogs) (5898) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:55,304::__init__::232::IOProcess::(_processLogs) (5898) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:55,304::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:55,305::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:55,306::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:55,306::__init__::232::IOProcess::(_processLogs) (5899) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:55,306::__init__::232::IOProcess::(_processLogs) (5899) Queuing response >Thread-14::DEBUG::2014-08-11 14:52:55,306::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:52:55,307::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:52:55,308::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:52:55,308::__init__::232::IOProcess::(_processLogs) (5900) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:52:55,309::__init__::232::IOProcess::(_processLogs) (5900) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:55,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:55,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:52:55,890::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:52:56,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:56,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130251 s, 78.6 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:52:56,766::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:52:56,767::task::595::Storage.TaskManager.Task::(_updateState) Task=`a169413b-e362-4067-8ec0-b8e681b02e24`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:52:56,767::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:52:56,780::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:56,781::task::1191::Storage.TaskManager.Task::(prepare) Task=`a169413b-e362-4067-8ec0-b8e681b02e24`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:52:56,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`a169413b-e362-4067-8ec0-b8e681b02e24`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:52:56,781::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:52:56,781::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:52:56,781::task::993::Storage.TaskManager.Task::(_decref) Task=`a169413b-e362-4067-8ec0-b8e681b02e24`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:52:56,792::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:52:56,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bcf712d-6608-4766-baff-6f28c91c4bce`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:52:56,793::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:52:56,794::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d4ceaa21-7cae-45fd-a8a3-88f14b637f55`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:52:56,794::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:52:56,795::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:52:56,795::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d4ceaa21-7cae-45fd-a8a3-88f14b637f55`::Granted request >Thread-34::DEBUG::2014-08-11 14:52:56,796::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8bcf712d-6608-4766-baff-6f28c91c4bce`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:52:56,796::task::993::Storage.TaskManager.Task::(_decref) Task=`8bcf712d-6608-4766-baff-6f28c91c4bce`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:52:56,816::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859524362240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:56,816::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bcf712d-6608-4766-baff-6f28c91c4bce`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859524362240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:52:56,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bcf712d-6608-4766-baff-6f28c91c4bce`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:52:56,816::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:52:56,817::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:52:56,817::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:52:56,817::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:52:56,817::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:52:56,818::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:52:56,818::task::993::Storage.TaskManager.Task::(_decref) Task=`8bcf712d-6608-4766-baff-6f28c91c4bce`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:52:58,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:52:58,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132058 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:00,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:00,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132651 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:00,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:00,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:00,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f8b53f9-67ca-402c-997f-d6198c40765f`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:00,982::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:53:00,984::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426211', 'lastCheck': '5.7', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:00,985::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:00,986::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f8b53f9-67ca-402c-997f-d6198c40765f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426211', 'lastCheck': '5.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:53:00,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f8b53f9-67ca-402c-997f-d6198c40765f`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:00,988::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:00,989::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:00,990::task::993::Storage.TaskManager.Task::(_decref) Task=`5f8b53f9-67ca-402c-997f-d6198c40765f`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:53:01,014::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:02,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:02,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266928 s, 38.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:04,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:04,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132719 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:05,309::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:05,310::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:53:05,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:53:05,312::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:05,330::__init__::232::IOProcess::(_processLogs) (5901) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:05,331::__init__::232::IOProcess::(_processLogs) (5901) Queuing response >Thread-16::DEBUG::2014-08-11 14:53:05,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000669818 s, 679 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:05,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:05,334::__init__::232::IOProcess::(_processLogs) (5902) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:05,335::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:05,335::__init__::232::IOProcess::(_processLogs) (5903) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:05,336::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:05,336::__init__::232::IOProcess::(_processLogs) (5904) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:05,337::__init__::232::IOProcess::(_processLogs) (5904) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:05,338::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:05,338::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:05,339::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:05,340::__init__::232::IOProcess::(_processLogs) (5905) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:05,340::__init__::232::IOProcess::(_processLogs) (5905) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:05,996::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:05,997::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:05,998::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:06,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:06,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132043 s, 77.6 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:53:06,830::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:53:06,831::task::595::Storage.TaskManager.Task::(_updateState) Task=`c686e46d-e496-4750-aed1-d63c3ec9e89a`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:06,831::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:53:06,844::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:06,845::task::1191::Storage.TaskManager.Task::(prepare) Task=`c686e46d-e496-4750-aed1-d63c3ec9e89a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:06,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`c686e46d-e496-4750-aed1-d63c3ec9e89a`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:06,845::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:06,845::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:06,846::task::993::Storage.TaskManager.Task::(_decref) Task=`c686e46d-e496-4750-aed1-d63c3ec9e89a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:53:06,857::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:53:06,857::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e0c40f9-2664-4886-b7e2-9c0010aaf942`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:53:06,858::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:53:06,858::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`49656733-421b-411b-96ac-06978404ed40`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:53:06,859::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:53:06,859::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:53:06,859::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`49656733-421b-411b-96ac-06978404ed40`::Granted request >Thread-34::DEBUG::2014-08-11 14:53:06,860::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9e0c40f9-2664-4886-b7e2-9c0010aaf942`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:53:06,860::task::993::Storage.TaskManager.Task::(_decref) Task=`9e0c40f9-2664-4886-b7e2-9c0010aaf942`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:53:06,880::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859511517184', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:06,880::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e0c40f9-2664-4886-b7e2-9c0010aaf942`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859511517184', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:06,880::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e0c40f9-2664-4886-b7e2-9c0010aaf942`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:53:06,881::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:53:06,881::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:53:06,881::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:53:06,882::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:53:06,882::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:53:06,882::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:53:06,882::task::993::Storage.TaskManager.Task::(_decref) Task=`9e0c40f9-2664-4886-b7e2-9c0010aaf942`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:08,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:08,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131933 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:10,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:10,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131682 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:11,005::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:11,006::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:11,081::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:12,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:12,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013179 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:53:13,770::task::595::Storage.TaskManager.Task::(_updateState) Task=`25d3ae7c-f7e8-4c0e-82c3-49934bf84a1e`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:53:13,770::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:53:13,771::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:13,771::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:13,772::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:13,772::__init__::232::IOProcess::(_processLogs) (5906) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:53:13,773::__init__::232::IOProcess::(_processLogs) DEBUG|(5906) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:13,774::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:13,774::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:13,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:13,775::__init__::232::IOProcess::(_processLogs) DEBUG|(5907) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:53:13,775::__init__::232::IOProcess::(_processLogs) (5907) Queuing response >Thread-55::INFO::2014-08-11 14:53:13,775::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:53:13,776::task::1191::Storage.TaskManager.Task::(prepare) Task=`25d3ae7c-f7e8-4c0e-82c3-49934bf84a1e`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:53:13,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`25d3ae7c-f7e8-4c0e-82c3-49934bf84a1e`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:53:13,776::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:53:13,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:53:13,777::task::993::Storage.TaskManager.Task::(_decref) Task=`25d3ae7c-f7e8-4c0e-82c3-49934bf84a1e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:14,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:14,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133067 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:15,340::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:15,341::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:15,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:15,342::__init__::232::IOProcess::(_processLogs) (5908) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:53:15,342::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:53:15,361::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421295 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:15,363::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:15,363::__init__::232::IOProcess::(_processLogs) (5909) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:15,364::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:15,365::__init__::232::IOProcess::(_processLogs) (5910) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:15,365::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5910) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:15,366::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:15,366::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:15,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:15,367::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5911) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:15,367::__init__::232::IOProcess::(_processLogs) (5911) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:15,368::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:15,368::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:15,368::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:15,369::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(5912) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:15,369::__init__::232::IOProcess::(_processLogs) (5912) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:16,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:16,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:16,090::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:16,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8348759-1120-4b7f-b280-e9fb0b3846bf`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:16,112::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:53:16,112::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421295', 'lastCheck': '0.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:53:16,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`d8348759-1120-4b7f-b280-e9fb0b3846bf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421295', 'lastCheck': '0.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:53:16,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8348759-1120-4b7f-b280-e9fb0b3846bf`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:16,113::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:16,113::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:16,113::task::993::Storage.TaskManager.Task::(_decref) Task=`d8348759-1120-4b7f-b280-e9fb0b3846bf`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:53:16,132::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:16,895::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:53:16,896::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d57fbd9-25e2-4397-97a2-5a473ede1e76`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:16,896::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:53:16,909::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:16,909::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d57fbd9-25e2-4397-97a2-5a473ede1e76`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:16,909::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d57fbd9-25e2-4397-97a2-5a473ede1e76`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:16,909::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:16,910::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:16,910::task::993::Storage.TaskManager.Task::(_decref) Task=`6d57fbd9-25e2-4397-97a2-5a473ede1e76`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:16,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::DEBUG::2014-08-11 14:53:16,937::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:53:16,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`58c07172-18c9-4451-8867-a1906767f72b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:53:16,940::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:53:16,942::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a5169dc2-07a4-4976-9a07-1f73bf6873d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:53:16,942::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:53:16,944::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:53:16,944::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a5169dc2-07a4-4976-9a07-1f73bf6873d1`::Granted request >Thread-34::DEBUG::2014-08-11 14:53:16,945::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`58c07172-18c9-4451-8867-a1906767f72b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:53:16,945::task::993::Storage.TaskManager.Task::(_decref) Task=`58c07172-18c9-4451-8867-a1906767f72b`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:16,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0404384 s, 25.3 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 14:53:17,009::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859504373760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:17,011::task::1191::Storage.TaskManager.Task::(prepare) Task=`58c07172-18c9-4451-8867-a1906767f72b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859504373760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:17,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`58c07172-18c9-4451-8867-a1906767f72b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:53:17,013::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:53:17,014::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:53:17,015::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:53:17,017::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:53:17,018::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:53:17,018::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:53:17,020::task::993::Storage.TaskManager.Task::(_decref) Task=`58c07172-18c9-4451-8867-a1906767f72b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:19,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:19,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132961 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:21,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:21,103::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:21,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132161 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:21,185::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:21,189::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:23,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:23,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132098 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:25,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:25,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0194713 s, 52.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:25,371::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:25,371::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:25,372::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:25,372::__init__::232::IOProcess::(_processLogs) (5913) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:25,372::__init__::232::IOProcess::(_processLogs) (5913) Queuing response >Thread-16::DEBUG::2014-08-11 14:53:25,373::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:53:25,392::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428143 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:25,393::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:25,394::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:25,394::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:25,394::__init__::232::IOProcess::(_processLogs) (5914) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:25,395::__init__::232::IOProcess::(_processLogs) (5914) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:25,396::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:25,397::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:25,398::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:25,398::__init__::232::IOProcess::(_processLogs) (5915) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:25,398::__init__::232::IOProcess::(_processLogs) (5915) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:25,398::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:25,399::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:25,399::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:25,399::__init__::232::IOProcess::(_processLogs) (5916) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:25,400::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5916) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:25,401::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:25,401::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:25,402::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:25,403::__init__::232::IOProcess::(_processLogs) (5917) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:25,403::__init__::232::IOProcess::(_processLogs) (5917) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:26,198::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:26,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:26,282::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:27,036::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:53:27,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`84db14a8-72a5-45f8-9a6a-d5a70c263da7`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:27,037::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:53:27,050::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:27,050::task::1191::Storage.TaskManager.Task::(prepare) Task=`84db14a8-72a5-45f8-9a6a-d5a70c263da7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:27,050::task::595::Storage.TaskManager.Task::(_updateState) Task=`84db14a8-72a5-45f8-9a6a-d5a70c263da7`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:27,050::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:27,051::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:27,051::task::993::Storage.TaskManager.Task::(_decref) Task=`84db14a8-72a5-45f8-9a6a-d5a70c263da7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:53:27,062::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:53:27,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`146c9e00-240e-4e97-b16c-841e45f41e4a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:53:27,063::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:53:27,064::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cbb955e2-76d4-4a57-9e2d-d7da3cde2474`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:53:27,064::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:53:27,064::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:53:27,065::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cbb955e2-76d4-4a57-9e2d-d7da3cde2474`::Granted request >Thread-34::DEBUG::2014-08-11 14:53:27,065::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`146c9e00-240e-4e97-b16c-841e45f41e4a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:53:27,065::task::993::Storage.TaskManager.Task::(_decref) Task=`146c9e00-240e-4e97-b16c-841e45f41e4a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:53:27,085::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859492511744', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:27,086::task::1191::Storage.TaskManager.Task::(prepare) Task=`146c9e00-240e-4e97-b16c-841e45f41e4a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859492511744', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:27,086::task::595::Storage.TaskManager.Task::(_updateState) Task=`146c9e00-240e-4e97-b16c-841e45f41e4a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:53:27,086::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:53:27,086::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:53:27,087::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:53:27,087::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:53:27,087::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:53:27,087::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:53:27,088::task::993::Storage.TaskManager.Task::(_decref) Task=`146c9e00-240e-4e97-b16c-841e45f41e4a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:27,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:27,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131443 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:29,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:29,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132033 s, 77.6 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:53:31,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cb6b0c7-a7da-4d71-a3c3-ddf80180fecb`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:31,223::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:53:31,281::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428143', 'lastCheck': '5.9', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:53:31,281::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cb6b0c7-a7da-4d71-a3c3-ddf80180fecb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428143', 'lastCheck': '5.9', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:53:31,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cb6b0c7-a7da-4d71-a3c3-ddf80180fecb`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:31,281::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:31,282::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:31,282::task::993::Storage.TaskManager.Task::(_decref) Task=`3cb6b0c7-a7da-4d71-a3c3-ddf80180fecb`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:31,289::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:31,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:31,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:31,314::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:31,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:31,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133016 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:33,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:33,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131394 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:35,403::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:35,404::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:35,404::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:35,404::__init__::232::IOProcess::(_processLogs) (5918) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:35,404::__init__::232::IOProcess::(_processLogs) (5918) Queuing response >Thread-16::DEBUG::2014-08-11 14:53:35,405::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:53:35,424::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000429117 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:35,425::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:35,425::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:35,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:35,427::__init__::232::IOProcess::(_processLogs) (5919) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:35,427::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5919) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:35,429::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:35,430::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:35,430::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:35,430::__init__::232::IOProcess::(_processLogs) (5920) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:35,431::__init__::232::IOProcess::(_processLogs) (5920) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:35,431::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:35,432::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:35,432::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:35,433::__init__::232::IOProcess::(_processLogs) (5921) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:35,433::__init__::232::IOProcess::(_processLogs) (5921) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:35,433::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:35,434::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:35,434::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:35,436::__init__::232::IOProcess::(_processLogs) (5922) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:35,450::__init__::232::IOProcess::(_processLogs) (5922) Queuing response >Dummy-33::DEBUG::2014-08-11 14:53:35,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:35,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132333 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:36,298::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:36,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:36,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:37,100::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:53:37,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`accb1614-c5bb-4a71-af88-59f340cf87ca`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:37,102::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:53:37,115::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:37,115::task::1191::Storage.TaskManager.Task::(prepare) Task=`accb1614-c5bb-4a71-af88-59f340cf87ca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:37,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`accb1614-c5bb-4a71-af88-59f340cf87ca`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:37,115::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:37,116::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:37,116::task::993::Storage.TaskManager.Task::(_decref) Task=`accb1614-c5bb-4a71-af88-59f340cf87ca`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:53:37,126::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:53:37,127::task::595::Storage.TaskManager.Task::(_updateState) Task=`73c18426-84dd-4a52-841b-60e015c6756d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:53:37,127::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:53:37,128::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`32cff280-3f92-45ae-ae4f-6f9dbfcf5211`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:53:37,128::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:53:37,128::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:53:37,129::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`32cff280-3f92-45ae-ae4f-6f9dbfcf5211`::Granted request >Thread-34::DEBUG::2014-08-11 14:53:37,129::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`73c18426-84dd-4a52-841b-60e015c6756d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:53:37,129::task::993::Storage.TaskManager.Task::(_decref) Task=`73c18426-84dd-4a52-841b-60e015c6756d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:53:37,150::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859460857856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:37,150::task::1191::Storage.TaskManager.Task::(prepare) Task=`73c18426-84dd-4a52-841b-60e015c6756d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859460857856', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:37,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`73c18426-84dd-4a52-841b-60e015c6756d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:53:37,151::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:53:37,151::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:53:37,151::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:53:37,151::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:53:37,152::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:53:37,152::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:53:37,152::task::993::Storage.TaskManager.Task::(_decref) Task=`73c18426-84dd-4a52-841b-60e015c6756d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:37,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:37,670::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207566 s, 49.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:39,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:39,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133091 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:41,305::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:41,307::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:41,308::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:41,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:41,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132099 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:43,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:43,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131889 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:45,437::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:45,438::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:45,438::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:45,438::__init__::232::IOProcess::(_processLogs) (5923) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:45,439::__init__::232::IOProcess::(_processLogs) (5923) Queuing response >Thread-16::DEBUG::2014-08-11 14:53:45,439::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:53:45,458::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000438316 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:45,459::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:45,459::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:45,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:45,460::__init__::232::IOProcess::(_processLogs) (5924) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:45,461::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5924) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:45,462::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:45,463::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:45,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:45,463::__init__::232::IOProcess::(_processLogs) (5925) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:45,464::__init__::232::IOProcess::(_processLogs) (5925) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:45,464::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:45,465::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:45,465::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:45,465::__init__::232::IOProcess::(_processLogs) (5926) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:45,466::__init__::232::IOProcess::(_processLogs) (5926) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:45,466::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:45,467::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:45,467::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:45,468::__init__::232::IOProcess::(_processLogs) (5927) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:45,469::__init__::232::IOProcess::(_processLogs) (5927) Queuing response >Dummy-33::DEBUG::2014-08-11 14:53:45,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:45,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132568 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:46,316::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:46,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:46,382::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:46,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`205c9fbc-d670-4acb-b4ad-721c20c73004`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:46,408::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:53:46,408::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000438316', 'lastCheck': '0.9', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:53:46,409::task::1191::Storage.TaskManager.Task::(prepare) Task=`205c9fbc-d670-4acb-b4ad-721c20c73004`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000438316', 'lastCheck': '0.9', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:53:46,409::task::595::Storage.TaskManager.Task::(_updateState) Task=`205c9fbc-d670-4acb-b4ad-721c20c73004`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:46,409::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:46,409::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:46,410::task::993::Storage.TaskManager.Task::(_decref) Task=`205c9fbc-d670-4acb-b4ad-721c20c73004`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:53:46,428::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:47,166::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:53:47,167::task::595::Storage.TaskManager.Task::(_updateState) Task=`e12b1a78-b0da-412b-84ea-50835a23adaf`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:47,167::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:53:47,180::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:47,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`e12b1a78-b0da-412b-84ea-50835a23adaf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:47,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`e12b1a78-b0da-412b-84ea-50835a23adaf`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:47,181::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:47,181::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:47,181::task::993::Storage.TaskManager.Task::(_decref) Task=`e12b1a78-b0da-412b-84ea-50835a23adaf`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:53:47,194::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:53:47,195::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dbe9652-50ab-4552-8a63-f6617d5e6efd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:53:47,195::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:53:47,196::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`82e7ea4c-195d-4ca0-a3ed-f6f3f69ae0c5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:53:47,196::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:53:47,197::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:53:47,197::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`82e7ea4c-195d-4ca0-a3ed-f6f3f69ae0c5`::Granted request >Thread-34::DEBUG::2014-08-11 14:53:47,197::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9dbe9652-50ab-4552-8a63-f6617d5e6efd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:53:47,198::task::993::Storage.TaskManager.Task::(_decref) Task=`9dbe9652-50ab-4552-8a63-f6617d5e6efd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:53:47,217::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859394666496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:47,218::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dbe9652-50ab-4552-8a63-f6617d5e6efd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859394666496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:47,218::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dbe9652-50ab-4552-8a63-f6617d5e6efd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:53:47,218::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:53:47,218::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:53:47,219::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:53:47,219::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:53:47,219::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:53:47,219::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:53:47,220::task::993::Storage.TaskManager.Task::(_decref) Task=`9dbe9652-50ab-4552-8a63-f6617d5e6efd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:47,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:47,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133144 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:49,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:50,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132982 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:51,389::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:51,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:51,481::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:53:52,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:52,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131505 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:53:54,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:54,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131622 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:55,469::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:55,470::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:55,470::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:55,470::__init__::232::IOProcess::(_processLogs) (5928) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:55,470::__init__::232::IOProcess::(_processLogs) (5928) Queuing response >Thread-16::DEBUG::2014-08-11 14:53:55,471::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:53:55,493::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418235 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:53:55,494::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:55,494::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:55,494::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:55,495::__init__::232::IOProcess::(_processLogs) (5929) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:53:55,495::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5929) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:55,497::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:55,497::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:55,498::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:55,498::__init__::232::IOProcess::(_processLogs) (5930) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:55,498::__init__::232::IOProcess::(_processLogs) (5930) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:55,499::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:55,499::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:55,500::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:55,501::__init__::232::IOProcess::(_processLogs) (5931) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:55,501::__init__::232::IOProcess::(_processLogs) (5931) Queuing response >Thread-14::DEBUG::2014-08-11 14:53:55,501::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:53:55,502::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:53:55,502::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:53:55,503::__init__::232::IOProcess::(_processLogs) (5932) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:53:55,503::__init__::232::IOProcess::(_processLogs) (5932) Queuing response >Dummy-33::DEBUG::2014-08-11 14:53:56,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:56,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132553 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:56,488::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:56,489::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:53:56,490::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:53:57,232::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:53:57,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`06d6c4f5-fb59-4e0b-90c6-f80d267822fc`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:53:57,233::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:53:57,246::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:57,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`06d6c4f5-fb59-4e0b-90c6-f80d267822fc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:53:57,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`06d6c4f5-fb59-4e0b-90c6-f80d267822fc`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:53:57,247::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:53:57,247::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:53:57,247::task::993::Storage.TaskManager.Task::(_decref) Task=`06d6c4f5-fb59-4e0b-90c6-f80d267822fc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:53:57,258::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:53:57,258::task::595::Storage.TaskManager.Task::(_updateState) Task=`18936b85-d346-40d2-ba7e-fbe0c682cdde`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:53:57,259::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:53:57,259::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3c5d255e-6e5b-4a5c-a598-79e9e25ef89c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:53:57,259::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:53:57,260::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:53:57,260::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3c5d255e-6e5b-4a5c-a598-79e9e25ef89c`::Granted request >Thread-34::DEBUG::2014-08-11 14:53:57,261::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`18936b85-d346-40d2-ba7e-fbe0c682cdde`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:53:57,261::task::993::Storage.TaskManager.Task::(_decref) Task=`18936b85-d346-40d2-ba7e-fbe0c682cdde`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:53:57,281::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326902272', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:57,281::task::1191::Storage.TaskManager.Task::(prepare) Task=`18936b85-d346-40d2-ba7e-fbe0c682cdde`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326902272', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:53:57,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`18936b85-d346-40d2-ba7e-fbe0c682cdde`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:53:57,282::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:53:57,282::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:53:57,282::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:53:57,282::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:53:57,283::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:53:57,283::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:53:57,283::task::993::Storage.TaskManager.Task::(_decref) Task=`18936b85-d346-40d2-ba7e-fbe0c682cdde`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:53:58,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:53:58,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132458 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:00,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:00,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132544 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:01,493::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:01,494::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:01,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f24df62-1f24-4996-8829-02d070ebd907`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:01,581::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:54:01,584::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418235', 'lastCheck': '6.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:01,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:01,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f24df62-1f24-4996-8829-02d070ebd907`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418235', 'lastCheck': '6.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:54:01,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f24df62-1f24-4996-8829-02d070ebd907`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:01,588::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:01,589::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:01,589::task::993::Storage.TaskManager.Task::(_decref) Task=`0f24df62-1f24-4996-8829-02d070ebd907`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:54:01,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:02,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:02,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0287248 s, 35.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:04,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:04,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133106 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:05,504::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:05,505::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:54:05,506::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:54:05,506::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:05,524::__init__::232::IOProcess::(_processLogs) (5933) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:05,525::__init__::232::IOProcess::(_processLogs) (5933) Queuing response >Thread-16::DEBUG::2014-08-11 14:54:05,526::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000564588 s, 806 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:05,527::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:05,527::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:05,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:05,528::__init__::232::IOProcess::(_processLogs) (5934) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:05,529::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5934) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:05,530::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:05,531::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:05,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:05,531::__init__::232::IOProcess::(_processLogs) (5935) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:05,532::__init__::232::IOProcess::(_processLogs) (5935) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:05,532::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:05,533::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:05,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:05,534::__init__::232::IOProcess::(_processLogs) (5936) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:05,534::__init__::232::IOProcess::(_processLogs) (5936) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:05,534::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:05,535::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:05,535::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:05,536::__init__::232::IOProcess::(_processLogs) (5937) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:05,536::__init__::232::IOProcess::(_processLogs) (5937) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:06,592::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:06,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:06,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:06,627::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:06,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131765 s, 77.7 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:54:07,297::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:54:07,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d50a77a-4975-431b-904c-b7f9423266cf`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:07,299::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:54:07,312::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:07,313::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d50a77a-4975-431b-904c-b7f9423266cf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:07,313::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d50a77a-4975-431b-904c-b7f9423266cf`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:07,313::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:07,313::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:07,313::task::993::Storage.TaskManager.Task::(_decref) Task=`1d50a77a-4975-431b-904c-b7f9423266cf`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:54:07,323::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:54:07,324::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d463196-54fe-4a64-b05b-eb2c164d8d89`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:54:07,324::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:54:07,325::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f23d3c09-bac3-488e-9e33-1ddbc21c56a4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:54:07,325::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:54:07,326::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:54:07,326::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f23d3c09-bac3-488e-9e33-1ddbc21c56a4`::Granted request >Thread-34::DEBUG::2014-08-11 14:54:07,327::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8d463196-54fe-4a64-b05b-eb2c164d8d89`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:54:07,327::task::993::Storage.TaskManager.Task::(_decref) Task=`8d463196-54fe-4a64-b05b-eb2c164d8d89`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:54:07,346::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326836736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:07,347::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d463196-54fe-4a64-b05b-eb2c164d8d89`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326836736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:07,347::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d463196-54fe-4a64-b05b-eb2c164d8d89`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:54:07,347::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:54:07,348::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:54:07,348::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:54:07,348::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:54:07,348::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:54:07,349::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:54:07,349::task::993::Storage.TaskManager.Task::(_decref) Task=`8d463196-54fe-4a64-b05b-eb2c164d8d89`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:54:08,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:08,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148631 s, 68.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:10,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:10,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131428 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:11,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:11,603::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:11,605::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:12,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:12,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131873 s, 77.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:54:13,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`377f8a54-0b65-4865-943e-c19964bc2261`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:54:13,900::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:54:13,901::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:13,901::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:13,902::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:13,902::__init__::232::IOProcess::(_processLogs) (5938) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:54:13,902::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5938) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:13,903::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:13,904::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:54:13,904::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:54:13,905::task::1191::Storage.TaskManager.Task::(prepare) Task=`377f8a54-0b65-4865-943e-c19964bc2261`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:54:13,905::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:54:13,906::task::595::Storage.TaskManager.Task::(_updateState) Task=`377f8a54-0b65-4865-943e-c19964bc2261`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:54:13,907::__init__::232::IOProcess::(_processLogs) (5939) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:54:13,907::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 14:54:13,908::__init__::232::IOProcess::(_processLogs) (5939) Queuing response >Thread-55::DEBUG::2014-08-11 14:54:13,909::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:54:13,909::task::993::Storage.TaskManager.Task::(_decref) Task=`377f8a54-0b65-4865-943e-c19964bc2261`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:54:14,849::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:14,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132941 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:15,537::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:15,537::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:15,538::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:54:15,539::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:54:15,540::__init__::232::IOProcess::(_processLogs) (5940) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:15,557::__init__::232::IOProcess::(_processLogs) (5940) Queuing response >Thread-16::DEBUG::2014-08-11 14:54:15,558::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000484057 s, 940 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:15,559::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:15,560::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:15,560::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:15,560::__init__::232::IOProcess::(_processLogs) (5941) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:15,560::__init__::232::IOProcess::(_processLogs) (5941) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:15,561::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:15,562::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:15,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:15,563::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5942) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:15,563::__init__::232::IOProcess::(_processLogs) (5942) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:15,564::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:15,564::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:15,565::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:15,565::__init__::232::IOProcess::(_processLogs) (5943) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:15,565::__init__::232::IOProcess::(_processLogs) (5943) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:15,566::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:15,566::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:15,566::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:15,567::__init__::232::IOProcess::(_processLogs) (5944) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:15,567::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5944) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:16,611::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:16,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:16,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:16,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb213917-473f-4fb7-9e3d-a3ca11a7e24b`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:16,699::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:54:16,699::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000484057', 'lastCheck': '1.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:54:16,700::task::1191::Storage.TaskManager.Task::(prepare) Task=`cb213917-473f-4fb7-9e3d-a3ca11a7e24b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000484057', 'lastCheck': '1.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:54:16,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb213917-473f-4fb7-9e3d-a3ca11a7e24b`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:16,700::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:16,700::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:16,700::task::993::Storage.TaskManager.Task::(_decref) Task=`cb213917-473f-4fb7-9e3d-a3ca11a7e24b`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:54:16,719::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:16,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:16,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131334 s, 78.0 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:54:17,361::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:54:17,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`22189724-2b71-48a9-8441-d39a499c5635`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:17,362::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:54:17,375::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:17,375::task::1191::Storage.TaskManager.Task::(prepare) Task=`22189724-2b71-48a9-8441-d39a499c5635`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:17,376::task::595::Storage.TaskManager.Task::(_updateState) Task=`22189724-2b71-48a9-8441-d39a499c5635`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:17,376::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:17,376::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:17,376::task::993::Storage.TaskManager.Task::(_decref) Task=`22189724-2b71-48a9-8441-d39a499c5635`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:54:17,387::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:54:17,387::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1f98672-64b5-41ec-ac0f-63e0d7046758`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:54:17,388::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:54:17,389::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`115097df-3e8e-4d0b-b4db-68196ee82a66`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:54:17,389::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:54:17,389::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:54:17,389::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`115097df-3e8e-4d0b-b4db-68196ee82a66`::Granted request >Thread-34::DEBUG::2014-08-11 14:54:17,390::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c1f98672-64b5-41ec-ac0f-63e0d7046758`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:54:17,390::task::993::Storage.TaskManager.Task::(_decref) Task=`c1f98672-64b5-41ec-ac0f-63e0d7046758`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:54:17,410::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326836736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:17,410::task::1191::Storage.TaskManager.Task::(prepare) Task=`c1f98672-64b5-41ec-ac0f-63e0d7046758`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326836736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:17,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1f98672-64b5-41ec-ac0f-63e0d7046758`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:54:17,411::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:54:17,411::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:54:17,411::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:54:17,412::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:54:17,412::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:54:17,412::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:54:17,412::task::993::Storage.TaskManager.Task::(_decref) Task=`c1f98672-64b5-41ec-ac0f-63e0d7046758`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:54:18,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:18,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135689 s, 75.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:21,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:21,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013346 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:21,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:21,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:21,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:23,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:23,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131838 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:25,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:25,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132544 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:25,569::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:25,569::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:25,569::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:25,570::__init__::232::IOProcess::(_processLogs) (5945) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:25,570::__init__::232::IOProcess::(_processLogs) (5945) Queuing response >Thread-16::DEBUG::2014-08-11 14:54:25,571::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:54:25,589::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420552 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:25,591::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:25,591::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:25,591::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:25,591::__init__::232::IOProcess::(_processLogs) (5946) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:25,592::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5946) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:25,593::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:25,594::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:25,595::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:25,595::__init__::232::IOProcess::(_processLogs) (5947) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:25,595::__init__::232::IOProcess::(_processLogs) (5947) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:25,595::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:25,596::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:25,596::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:25,596::__init__::232::IOProcess::(_processLogs) (5948) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:25,597::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|(5948) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:25,598::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:25,598::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:25,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:25,599::__init__::232::IOProcess::(_processLogs) (5949) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:25,600::__init__::232::IOProcess::(_processLogs) (5949) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:26,704::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:26,710::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:26,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:27,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:27,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132886 s, 77.1 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:54:27,425::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:54:27,426::task::595::Storage.TaskManager.Task::(_updateState) Task=`26eb6f36-8c39-4bfc-b9a8-ec12b9925e7b`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:27,426::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:54:27,439::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:27,439::task::1191::Storage.TaskManager.Task::(prepare) Task=`26eb6f36-8c39-4bfc-b9a8-ec12b9925e7b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:27,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`26eb6f36-8c39-4bfc-b9a8-ec12b9925e7b`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:27,440::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:27,440::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:27,440::task::993::Storage.TaskManager.Task::(_decref) Task=`26eb6f36-8c39-4bfc-b9a8-ec12b9925e7b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:54:27,451::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:54:27,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`859d48a1-871f-4006-a0a1-69093b7a745d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:54:27,452::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:54:27,452::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`36ee7b33-284c-4c61-b67b-a08ca7b2a442`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:54:27,453::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:54:27,453::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:54:27,453::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`36ee7b33-284c-4c61-b67b-a08ca7b2a442`::Granted request >Thread-34::DEBUG::2014-08-11 14:54:27,454::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`859d48a1-871f-4006-a0a1-69093b7a745d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:54:27,454::task::993::Storage.TaskManager.Task::(_decref) Task=`859d48a1-871f-4006-a0a1-69093b7a745d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:54:27,474::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326836736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:27,474::task::1191::Storage.TaskManager.Task::(prepare) Task=`859d48a1-871f-4006-a0a1-69093b7a745d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326836736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:27,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`859d48a1-871f-4006-a0a1-69093b7a745d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:54:27,475::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:54:27,475::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:54:27,475::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:54:27,476::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:54:27,476::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:54:27,476::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:54:27,476::task::993::Storage.TaskManager.Task::(_decref) Task=`859d48a1-871f-4006-a0a1-69093b7a745d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:54:29,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:29,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132406 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:31,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:31,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0179433 s, 57.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:31,791::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:31,792::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:31,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac97862-42b3-41fd-8a7b-f9c4275792ec`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:31,884::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:54:31,886::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420552', 'lastCheck': '6.3', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:54:31,887::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ac97862-42b3-41fd-8a7b-f9c4275792ec`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420552', 'lastCheck': '6.3', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:31,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:31,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ac97862-42b3-41fd-8a7b-f9c4275792ec`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:31,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:31,891::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:31,892::task::993::Storage.TaskManager.Task::(_decref) Task=`0ac97862-42b3-41fd-8a7b-f9c4275792ec`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:54:31,912::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:33,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:33,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132457 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:35,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:35,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132727 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:35,600::__init__::232::IOProcess::(_processLogs) (5914) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:35,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:35,602::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 14:54:35,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:54:35,603::__init__::232::IOProcess::(_processLogs) (5950) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:35,604::__init__::232::IOProcess::(_processLogs) (5950) Queuing response >Thread-16::DEBUG::2014-08-11 14:54:35,622::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413408 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:35,623::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:35,624::__init__::232::IOProcess::(_processLogs) (5951) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:35,625::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:35,625::__init__::232::IOProcess::(_processLogs) (5952) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:35,626::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:35,626::__init__::232::IOProcess::(_processLogs) (5953) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:35,627::__init__::232::IOProcess::(_processLogs) (5953) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:35,628::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:35,628::__init__::232::IOProcess::(_processLogs) (5954) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:35,629::__init__::232::IOProcess::(_processLogs) (5954) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:36,898::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:36,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:36,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:37,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-103::DEBUG::2014-08-11 14:54:37,504::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-103::DEBUG::2014-08-11 14:54:37,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`22b1a452-a4e7-4cf5-8058-3bc15ab512b7`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:37,506::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 14:54:37,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138939 s, 73.7 MB/s\n'; <rc> = 0 >Thread-103::INFO::2014-08-11 14:54:37,527::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:37,528::task::1191::Storage.TaskManager.Task::(prepare) Task=`22b1a452-a4e7-4cf5-8058-3bc15ab512b7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:37,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`22b1a452-a4e7-4cf5-8058-3bc15ab512b7`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:37,530::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:37,531::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:37,531::task::993::Storage.TaskManager.Task::(_decref) Task=`22b1a452-a4e7-4cf5-8058-3bc15ab512b7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:54:37,545::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-34::DEBUG::2014-08-11 14:54:37,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`3224e4de-c7db-4f9d-93e7-7262a735f8c5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:54:37,548::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:54:37,551::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f9d834c3-6307-440a-adae-8413f9a165b2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:54:37,552::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:54:37,554::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:54:37,555::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f9d834c3-6307-440a-adae-8413f9a165b2`::Granted request >Thread-34::DEBUG::2014-08-11 14:54:37,557::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3224e4de-c7db-4f9d-93e7-7262a735f8c5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:54:37,558::task::993::Storage.TaskManager.Task::(_decref) Task=`3224e4de-c7db-4f9d-93e7-7262a735f8c5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:54:37,605::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326574592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:37,605::task::1191::Storage.TaskManager.Task::(prepare) Task=`3224e4de-c7db-4f9d-93e7-7262a735f8c5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326574592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:37,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`3224e4de-c7db-4f9d-93e7-7262a735f8c5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:54:37,606::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:54:37,606::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:54:37,606::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:54:37,607::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:54:37,607::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:54:37,607::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:54:37,607::task::993::Storage.TaskManager.Task::(_decref) Task=`3224e4de-c7db-4f9d-93e7-7262a735f8c5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:54:39,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:39,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132386 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:41,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:41,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131442 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:41,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:41,908::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:41,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:43,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:43,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131745 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:45,630::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:45,631::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:45,631::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:45,631::__init__::232::IOProcess::(_processLogs) (5955) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:45,632::__init__::232::IOProcess::(_processLogs) (5955) Queuing response >Thread-16::DEBUG::2014-08-11 14:54:45,632::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:54:45,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423773 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:45,652::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:45,652::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:45,652::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:45,653::__init__::232::IOProcess::(_processLogs) (5956) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:45,653::__init__::232::IOProcess::(_processLogs) (5956) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:45,654::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:45,655::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:45,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:45,656::__init__::232::IOProcess::(_processLogs) (5957) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:45,656::__init__::232::IOProcess::(_processLogs) (5957) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:45,656::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:45,657::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:45,657::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:45,658::__init__::232::IOProcess::(_processLogs) (5958) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:45,658::__init__::232::IOProcess::(_processLogs) (5958) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:45,658::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:45,659::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:45,659::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:45,660::__init__::232::IOProcess::(_processLogs) (5959) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:45,661::__init__::232::IOProcess::(_processLogs) (5959) Queuing response >Dummy-33::DEBUG::2014-08-11 14:54:45,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:45,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131357 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:46,990::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:46,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:46,993::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:47,001::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3271c82-9d3d-44a8-af79-c64329448807`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:47,002::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:54:47,002::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423773', 'lastCheck': '1.3', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:54:47,003::task::1191::Storage.TaskManager.Task::(prepare) Task=`c3271c82-9d3d-44a8-af79-c64329448807`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423773', 'lastCheck': '1.3', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:54:47,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3271c82-9d3d-44a8-af79-c64329448807`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:47,003::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:47,003::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:47,003::task::993::Storage.TaskManager.Task::(_decref) Task=`c3271c82-9d3d-44a8-af79-c64329448807`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:54:47,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:47,619::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:54:47,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`e66e6a80-44e3-4022-ba4e-ca3a00b20856`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:47,620::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:54:47,633::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:47,633::task::1191::Storage.TaskManager.Task::(prepare) Task=`e66e6a80-44e3-4022-ba4e-ca3a00b20856`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:47,633::task::595::Storage.TaskManager.Task::(_updateState) Task=`e66e6a80-44e3-4022-ba4e-ca3a00b20856`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:47,634::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:47,634::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:47,634::task::993::Storage.TaskManager.Task::(_decref) Task=`e66e6a80-44e3-4022-ba4e-ca3a00b20856`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:54:47,645::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:54:47,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e358909-b121-49c3-a575-304e361ac540`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:54:47,646::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:54:47,647::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`79fe4f7e-0b9e-4832-8f94-cebc3be10f70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:54:47,647::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:54:47,647::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:54:47,647::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`79fe4f7e-0b9e-4832-8f94-cebc3be10f70`::Granted request >Thread-34::DEBUG::2014-08-11 14:54:47,648::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2e358909-b121-49c3-a575-304e361ac540`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:54:47,648::task::993::Storage.TaskManager.Task::(_decref) Task=`2e358909-b121-49c3-a575-304e361ac540`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:54:47,668::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326246912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:47,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e358909-b121-49c3-a575-304e361ac540`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859326246912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:47,669::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e358909-b121-49c3-a575-304e361ac540`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:54:47,669::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:54:47,669::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:54:47,669::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:54:47,670::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:54:47,670::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:54:47,670::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:54:47,670::task::993::Storage.TaskManager.Task::(_decref) Task=`2e358909-b121-49c3-a575-304e361ac540`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:54:47,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:47,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132098 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:49,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:49,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132957 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:54:51,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:51,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132404 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:51,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:52,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:52,008::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:54:54,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:54,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013206 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:55,661::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:55,662::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:55,662::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:55,662::__init__::232::IOProcess::(_processLogs) (5960) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:55,662::__init__::232::IOProcess::(_processLogs) (5960) Queuing response >Thread-16::DEBUG::2014-08-11 14:54:55,663::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:54:55,682::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000429484 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:54:55,683::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:55,683::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:55,684::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:55,684::__init__::232::IOProcess::(_processLogs) (5961) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:54:55,684::__init__::232::IOProcess::(_processLogs) (5961) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:55,685::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:55,686::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:55,687::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:55,687::__init__::232::IOProcess::(_processLogs) (5962) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:55,687::__init__::232::IOProcess::(_processLogs) (5962) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:55,687::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:55,688::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:55,689::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:55,689::__init__::232::IOProcess::(_processLogs) (5963) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:55,689::__init__::232::IOProcess::(_processLogs) (5963) Queuing response >Thread-14::DEBUG::2014-08-11 14:54:55,690::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:54:55,690::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:54:55,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:54:55,692::__init__::232::IOProcess::(_processLogs) (5964) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:54:55,692::__init__::232::IOProcess::(_processLogs) (5964) Queuing response >Dummy-33::DEBUG::2014-08-11 14:54:56,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:56,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013409 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:57,019::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:57,025::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:54:57,084::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:54:57,682::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:54:57,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`52f17b6b-8731-4a60-be81-e86645b0df11`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:54:57,684::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:54:57,696::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:57,697::task::1191::Storage.TaskManager.Task::(prepare) Task=`52f17b6b-8731-4a60-be81-e86645b0df11`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:54:57,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`52f17b6b-8731-4a60-be81-e86645b0df11`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:54:57,697::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:54:57,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:54:57,698::task::993::Storage.TaskManager.Task::(_decref) Task=`52f17b6b-8731-4a60-be81-e86645b0df11`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:54:57,708::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:54:57,709::task::595::Storage.TaskManager.Task::(_updateState) Task=`67049e65-14da-43ff-802b-84d56cfab165`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:54:57,709::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:54:57,710::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fd0647d4-76d9-4ca1-8c49-f6fae66ed5d0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:54:57,710::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:54:57,710::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:54:57,710::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fd0647d4-76d9-4ca1-8c49-f6fae66ed5d0`::Granted request >Thread-34::DEBUG::2014-08-11 14:54:57,711::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`67049e65-14da-43ff-802b-84d56cfab165`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:54:57,711::task::993::Storage.TaskManager.Task::(_decref) Task=`67049e65-14da-43ff-802b-84d56cfab165`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:54:57,731::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859181740032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:57,731::task::1191::Storage.TaskManager.Task::(prepare) Task=`67049e65-14da-43ff-802b-84d56cfab165`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1859181740032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:54:57,732::task::595::Storage.TaskManager.Task::(_updateState) Task=`67049e65-14da-43ff-802b-84d56cfab165`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:54:57,732::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:54:57,732::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:54:57,732::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:54:57,733::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:54:57,733::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:54:57,733::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:54:57,734::task::993::Storage.TaskManager.Task::(_decref) Task=`67049e65-14da-43ff-802b-84d56cfab165`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:54:58,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:54:58,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132695 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:00,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:00,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013348 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:02,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:02,097::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:02,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:02,122::task::595::Storage.TaskManager.Task::(_updateState) Task=`1021ca08-e976-4951-ab53-74f67fb04a66`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:02,124::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:55:02,125::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429484', 'lastCheck': '6.4', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:55:02,127::task::1191::Storage.TaskManager.Task::(prepare) Task=`1021ca08-e976-4951-ab53-74f67fb04a66`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429484', 'lastCheck': '6.4', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:55:02,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`1021ca08-e976-4951-ab53-74f67fb04a66`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:02,129::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:02,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:02,131::task::993::Storage.TaskManager.Task::(_decref) Task=`1021ca08-e976-4951-ab53-74f67fb04a66`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:55:02,176::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:55:02,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:02,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0310219 s, 33.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:04,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:04,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131697 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:05,692::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:05,693::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:05,693::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:05,693::__init__::232::IOProcess::(_processLogs) (5965) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:05,694::__init__::232::IOProcess::(_processLogs) DEBUG|(5965) Queuing response >Thread-16::DEBUG::2014-08-11 14:55:05,695::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:55:05,714::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422862 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:05,715::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:05,716::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:05,716::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:05,716::__init__::232::IOProcess::(_processLogs) (5966) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:05,717::__init__::232::IOProcess::(_processLogs) DEBUG|(5966) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:05,718::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:05,719::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:05,720::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:05,720::__init__::232::IOProcess::(_processLogs) (5967) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:05,720::__init__::232::IOProcess::(_processLogs) (5967) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:05,721::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:05,721::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:05,722::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:05,722::__init__::232::IOProcess::(_processLogs) (5968) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:05,722::__init__::232::IOProcess::(_processLogs) (5968) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:05,723::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:05,723::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:05,724::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:05,725::__init__::232::IOProcess::(_processLogs) (5969) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:05,725::__init__::232::IOProcess::(_processLogs) (5969) Queuing response >Dummy-33::DEBUG::2014-08-11 14:55:06,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:06,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01318 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:07,115::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:07,120::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:07,187::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:07,746::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:55:07,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7c0f68f-bec0-4be6-9218-4a01177019bc`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:07,748::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:55:07,761::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:07,762::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7c0f68f-bec0-4be6-9218-4a01177019bc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:07,762::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7c0f68f-bec0-4be6-9218-4a01177019bc`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:07,762::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:07,762::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:07,762::task::993::Storage.TaskManager.Task::(_decref) Task=`c7c0f68f-bec0-4be6-9218-4a01177019bc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:55:07,773::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:55:07,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fa0b9d9-26bf-4aeb-8a46-8685ce11fe78`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:55:07,774::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:55:07,775::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6280ca69-4ddf-408a-9ed7-712a556356b6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:55:07,775::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:55:07,775::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:55:07,775::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6280ca69-4ddf-408a-9ed7-712a556356b6`::Granted request >Thread-34::DEBUG::2014-08-11 14:55:07,776::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8fa0b9d9-26bf-4aeb-8a46-8685ce11fe78`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:55:07,776::task::993::Storage.TaskManager.Task::(_decref) Task=`8fa0b9d9-26bf-4aeb-8a46-8685ce11fe78`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:55:07,796::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858928050176', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:07,796::task::1191::Storage.TaskManager.Task::(prepare) Task=`8fa0b9d9-26bf-4aeb-8a46-8685ce11fe78`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858928050176', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:07,797::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fa0b9d9-26bf-4aeb-8a46-8685ce11fe78`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:55:07,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:55:07,797::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:55:07,797::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:55:07,798::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:55:07,798::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:55:07,798::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:55:07,798::task::993::Storage.TaskManager.Task::(_decref) Task=`8fa0b9d9-26bf-4aeb-8a46-8685ce11fe78`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:55:08,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:08,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132054 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:10,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:10,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133451 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:12,194::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:12,195::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:12,287::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:55:12,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:12,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132309 s, 77.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:55:14,017::task::595::Storage.TaskManager.Task::(_updateState) Task=`57f75fbf-684f-4322-bdb8-7e35eb9d4afb`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:55:14,018::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:55:14,019::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:14,019::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:14,019::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:14,019::__init__::232::IOProcess::(_processLogs) (5970) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:55:14,020::__init__::232::IOProcess::(_processLogs) DEBUG|(5970) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:14,021::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:14,021::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:55:14,022::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:55:14,023::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:55:14,024::task::1191::Storage.TaskManager.Task::(prepare) Task=`57f75fbf-684f-4322-bdb8-7e35eb9d4afb`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:55:14,024::__init__::232::IOProcess::(_processLogs) (5971) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:55:14,025::task::595::Storage.TaskManager.Task::(_updateState) Task=`57f75fbf-684f-4322-bdb8-7e35eb9d4afb`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:55:14,025::__init__::232::IOProcess::(_processLogs) (5971) Queuing response >Thread-55::DEBUG::2014-08-11 14:55:14,026::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:55:14,027::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:55:14,027::task::993::Storage.TaskManager.Task::(_decref) Task=`57f75fbf-684f-4322-bdb8-7e35eb9d4afb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:55:14,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:14,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132282 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:15,726::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:15,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:15,727::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:55:15,728::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:55:15,728::__init__::232::IOProcess::(_processLogs) (5972) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:15,746::__init__::232::IOProcess::(_processLogs) (5972) Queuing response >Thread-16::DEBUG::2014-08-11 14:55:15,747::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426278 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:15,748::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:55:15,749::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:15,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:15,749::__init__::232::IOProcess::(_processLogs) (5973) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:15,750::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:15,751::__init__::232::IOProcess::(_processLogs) (5974) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:15,752::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:15,752::__init__::232::IOProcess::(_processLogs) (5975) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:15,752::__init__::232::IOProcess::(_processLogs) (5975) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:15,753::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:15,754::__init__::232::IOProcess::(_processLogs) (5976) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:55:16,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:16,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016822 s, 60.9 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:55:17,283::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c7a10bc-9d92-4ad8-b044-09d593333c37`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:17,286::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:55:17,287::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426278', 'lastCheck': '1.5', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:55:17,287::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c7a10bc-9d92-4ad8-b044-09d593333c37`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426278', 'lastCheck': '1.5', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:55:17,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c7a10bc-9d92-4ad8-b044-09d593333c37`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:17,288::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:17,288::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:17,288::task::993::Storage.TaskManager.Task::(_decref) Task=`2c7a10bc-9d92-4ad8-b044-09d593333c37`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:17,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:17,298::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:17,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:17,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:17,810::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:55:17,811::task::595::Storage.TaskManager.Task::(_updateState) Task=`c522d807-c66c-453f-860c-d2a683af4440`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:17,811::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:55:17,824::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:17,824::task::1191::Storage.TaskManager.Task::(prepare) Task=`c522d807-c66c-453f-860c-d2a683af4440`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:17,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`c522d807-c66c-453f-860c-d2a683af4440`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:17,824::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:17,825::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:17,825::task::993::Storage.TaskManager.Task::(_decref) Task=`c522d807-c66c-453f-860c-d2a683af4440`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:55:17,836::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:55:17,836::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fe0192e-1039-4498-bc2f-cb4373eb4126`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:55:17,837::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:55:17,837::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9d183bc2-7674-48fe-9398-6af849caa064`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:55:17,838::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:55:17,838::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:55:17,838::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9d183bc2-7674-48fe-9398-6af849caa064`::Granted request >Thread-34::DEBUG::2014-08-11 14:55:17,839::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7fe0192e-1039-4498-bc2f-cb4373eb4126`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:55:17,839::task::993::Storage.TaskManager.Task::(_decref) Task=`7fe0192e-1039-4498-bc2f-cb4373eb4126`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:55:17,859::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858927984640', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:17,859::task::1191::Storage.TaskManager.Task::(prepare) Task=`7fe0192e-1039-4498-bc2f-cb4373eb4126`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858927984640', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:17,859::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fe0192e-1039-4498-bc2f-cb4373eb4126`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:55:17,860::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:55:17,860::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:55:17,860::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:55:17,860::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:55:17,861::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:55:17,861::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:55:17,861::task::993::Storage.TaskManager.Task::(_decref) Task=`7fe0192e-1039-4498-bc2f-cb4373eb4126`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:55:18,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:18,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131215 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:20,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:20,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130557 s, 78.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:22,306::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:22,307::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:22,387::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:55:23,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:23,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132302 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:25,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:25,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131691 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:25,756::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:25,756::__init__::232::IOProcess::(_processLogs) (5977) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:55:25,757::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:55:25,776::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421096 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:25,777::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:25,777::__init__::232::IOProcess::(_processLogs) (5978) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:25,778::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:25,779::__init__::232::IOProcess::(_processLogs) (5979) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:25,780::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:25,780::__init__::232::IOProcess::(_processLogs) (5980) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:25,781::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:25,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:25,782::__init__::232::IOProcess::(_processLogs) (5981) Queuing response >Dummy-33::DEBUG::2014-08-11 14:55:27,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:27,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132318 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:27,394::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:27,395::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:27,396::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:27,873::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:55:27,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ef76dd8-4ed5-431c-93d0-cb635563fcf6`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:27,874::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:55:27,887::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:27,887::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ef76dd8-4ed5-431c-93d0-cb635563fcf6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:27,888::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ef76dd8-4ed5-431c-93d0-cb635563fcf6`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:27,888::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:27,888::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:27,888::task::993::Storage.TaskManager.Task::(_decref) Task=`3ef76dd8-4ed5-431c-93d0-cb635563fcf6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:55:27,898::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:55:27,899::task::595::Storage.TaskManager.Task::(_updateState) Task=`623bc9fb-beec-473f-9cd9-2317bb9fab40`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:55:27,899::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:55:27,900::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6904e8cd-3881-4ab1-8b26-837be462e4e9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:55:27,900::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:55:27,901::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:55:27,901::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6904e8cd-3881-4ab1-8b26-837be462e4e9`::Granted request >Thread-34::DEBUG::2014-08-11 14:55:27,901::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`623bc9fb-beec-473f-9cd9-2317bb9fab40`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:55:27,901::task::993::Storage.TaskManager.Task::(_decref) Task=`623bc9fb-beec-473f-9cd9-2317bb9fab40`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:55:27,921::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858922741760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:27,922::task::1191::Storage.TaskManager.Task::(prepare) Task=`623bc9fb-beec-473f-9cd9-2317bb9fab40`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858922741760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:27,922::task::595::Storage.TaskManager.Task::(_updateState) Task=`623bc9fb-beec-473f-9cd9-2317bb9fab40`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:55:27,922::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:55:27,922::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:55:27,923::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:55:27,923::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:55:27,923::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:55:27,923::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:55:27,924::task::993::Storage.TaskManager.Task::(_decref) Task=`623bc9fb-beec-473f-9cd9-2317bb9fab40`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:55:29,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:29,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132207 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:31,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:31,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132586 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:32,404::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:32,411::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:32,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`d214c311-7744-4ac3-b78f-ac8686cc6ef3`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:32,488::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:55:32,491::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421096', 'lastCheck': '6.7', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:32,492::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:32,493::task::1191::Storage.TaskManager.Task::(prepare) Task=`d214c311-7744-4ac3-b78f-ac8686cc6ef3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421096', 'lastCheck': '6.7', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:55:32,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`d214c311-7744-4ac3-b78f-ac8686cc6ef3`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:32,494::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:32,495::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:32,497::task::993::Storage.TaskManager.Task::(_decref) Task=`d214c311-7744-4ac3-b78f-ac8686cc6ef3`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:55:32,540::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:55:33,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:33,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132897 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:35,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:35,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131594 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:35,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:35,784::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:35,784::__init__::232::IOProcess::(_processLogs) (5982) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:55:35,785::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:55:35,804::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041909 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:35,805::__init__::232::IOProcess::(_processLogs) (5983) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:35,806::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:35,807::__init__::232::IOProcess::(_processLogs) (5984) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:35,808::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:35,808::__init__::232::IOProcess::(_processLogs) (5985) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:35,808::__init__::232::IOProcess::(_processLogs) (5985) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:35,809::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:35,809::__init__::232::IOProcess::(_processLogs) (5986) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:55:37,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:37,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0261811 s, 39.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:37,503::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:37,509::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:37,515::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:37,936::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:55:37,937::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee292176-66c6-400d-8958-014fdecd9e66`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:37,937::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:55:37,950::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:37,950::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee292176-66c6-400d-8958-014fdecd9e66`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:37,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee292176-66c6-400d-8958-014fdecd9e66`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:37,951::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:37,951::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:37,951::task::993::Storage.TaskManager.Task::(_decref) Task=`ee292176-66c6-400d-8958-014fdecd9e66`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:55:37,961::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:55:37,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd9ed239-7117-4832-b854-4a1aa4764336`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:55:37,962::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:55:37,963::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3c16bf58-0bba-4f1b-94d9-884dc9caec87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:55:37,963::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:55:37,963::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:55:37,964::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3c16bf58-0bba-4f1b-94d9-884dc9caec87`::Granted request >Thread-34::DEBUG::2014-08-11 14:55:37,964::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bd9ed239-7117-4832-b854-4a1aa4764336`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:55:37,964::task::993::Storage.TaskManager.Task::(_decref) Task=`bd9ed239-7117-4832-b854-4a1aa4764336`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:55:37,984::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858921824256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:37,985::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd9ed239-7117-4832-b854-4a1aa4764336`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858921824256', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:37,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd9ed239-7117-4832-b854-4a1aa4764336`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:55:37,985::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:55:37,985::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:55:37,986::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:55:37,986::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:55:37,986::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:55:37,986::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:55:37,987::task::993::Storage.TaskManager.Task::(_decref) Task=`bd9ed239-7117-4832-b854-4a1aa4764336`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:55:39,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:39,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131481 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:41,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:41,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131612 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:42,523::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:42,524::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:42,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:55:43,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:43,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132822 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:45,811::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:45,812::__init__::232::IOProcess::(_processLogs) (5987) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:55:45,812::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:45,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 14:55:45,848::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000508118 s, 895 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:45,850::__init__::232::IOProcess::(_processLogs) (5988) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:45,861::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:45,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:45,861::__init__::232::IOProcess::(_processLogs) (5989) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:45,862::__init__::232::IOProcess::(_processLogs) (5989) Queuing response >Dummy-33::DEBUG::2014-08-11 14:55:45,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191144 s, 53.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:45,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:45,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:45,874::__init__::232::IOProcess::(_processLogs) (5990) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:45,876::__init__::232::IOProcess::(_processLogs) (5990) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:45,879::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:55:45,880::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:45,882::__init__::232::IOProcess::(_processLogs) (5991) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:45,883::__init__::232::IOProcess::(_processLogs) (5991) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:47,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:47,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:47,645::task::595::Storage.TaskManager.Task::(_updateState) Task=`28c543e4-c1fc-42e1-a141-73a004370e82`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:47,688::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:55:47,690::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000508118', 'lastCheck': '1.8', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:47,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:47,693::task::1191::Storage.TaskManager.Task::(prepare) Task=`28c543e4-c1fc-42e1-a141-73a004370e82`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000508118', 'lastCheck': '1.8', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:55:47,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`28c543e4-c1fc-42e1-a141-73a004370e82`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:47,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:47,699::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:47,701::task::993::Storage.TaskManager.Task::(_decref) Task=`28c543e4-c1fc-42e1-a141-73a004370e82`::ref 0 aborting False >Thread-103::DEBUG::2014-08-11 14:55:47,748::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:55:47,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:47,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131482 s, 77.9 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:55:47,999::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:55:48,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`348ebe61-1e43-42a2-b791-852891918447`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:48,001::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:55:48,013::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:48,014::task::1191::Storage.TaskManager.Task::(prepare) Task=`348ebe61-1e43-42a2-b791-852891918447`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:48,014::task::595::Storage.TaskManager.Task::(_updateState) Task=`348ebe61-1e43-42a2-b791-852891918447`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:48,014::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:48,014::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:48,015::task::993::Storage.TaskManager.Task::(_decref) Task=`348ebe61-1e43-42a2-b791-852891918447`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:55:48,027::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:55:48,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`71fe68a3-2ce3-4e3f-8bb4-d9e08be2a297`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:55:48,028::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:55:48,029::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8d83671e-12dc-4825-9841-f1d65877f7a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:55:48,029::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:55:48,029::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:55:48,030::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8d83671e-12dc-4825-9841-f1d65877f7a1`::Granted request >Thread-34::DEBUG::2014-08-11 14:55:48,030::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`71fe68a3-2ce3-4e3f-8bb4-d9e08be2a297`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:55:48,030::task::993::Storage.TaskManager.Task::(_decref) Task=`71fe68a3-2ce3-4e3f-8bb4-d9e08be2a297`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:55:48,050::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858921562112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:48,051::task::1191::Storage.TaskManager.Task::(prepare) Task=`71fe68a3-2ce3-4e3f-8bb4-d9e08be2a297`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858921562112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:48,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`71fe68a3-2ce3-4e3f-8bb4-d9e08be2a297`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:55:48,051::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:55:48,051::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:55:48,052::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:55:48,052::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:55:48,052::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:55:48,052::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:55:48,053::task::993::Storage.TaskManager.Task::(_decref) Task=`71fe68a3-2ce3-4e3f-8bb4-d9e08be2a297`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:55:49,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:49,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132374 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:55:52,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:52,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131638 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:52,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:52,704::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:52,787::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:55:54,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:54,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156551 s, 65.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:55,881::__init__::232::IOProcess::(_processLogs) (5992) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:55:55,882::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:55:55,900::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421955 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:55:55,901::__init__::232::IOProcess::(_processLogs) (5993) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:55:55,902::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:55,903::__init__::232::IOProcess::(_processLogs) (5994) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:55,904::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:55,904::__init__::232::IOProcess::(_processLogs) (5995) Queuing response >Thread-14::DEBUG::2014-08-11 14:55:55,905::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:55:55,905::__init__::232::IOProcess::(_processLogs) (5996) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:55:55,906::__init__::232::IOProcess::(_processLogs) (5996) Queuing response >Dummy-33::DEBUG::2014-08-11 14:55:56,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:56,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132308 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:57,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:57,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:55:57,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:55:58,065::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:55:58,066::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6977e2b-d5a2-425c-9ca9-0adc9af2209a`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:55:58,066::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:55:58,079::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:58,079::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6977e2b-d5a2-425c-9ca9-0adc9af2209a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:55:58,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6977e2b-d5a2-425c-9ca9-0adc9af2209a`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:55:58,080::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:55:58,080::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:55:58,080::task::993::Storage.TaskManager.Task::(_decref) Task=`b6977e2b-d5a2-425c-9ca9-0adc9af2209a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:55:58,091::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:55:58,091::task::595::Storage.TaskManager.Task::(_updateState) Task=`d546e964-9366-4c4e-9143-f83d51e00e43`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:55:58,091::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:55:58,092::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2a6d76e1-a713-41ba-a589-978ca972526c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:55:58,092::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:55:58,093::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:55:58,093::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2a6d76e1-a713-41ba-a589-978ca972526c`::Granted request >Thread-34::DEBUG::2014-08-11 14:55:58,094::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d546e964-9366-4c4e-9143-f83d51e00e43`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:55:58,094::task::993::Storage.TaskManager.Task::(_decref) Task=`d546e964-9366-4c4e-9143-f83d51e00e43`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:55:58,114::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858921037824', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:58,115::task::1191::Storage.TaskManager.Task::(prepare) Task=`d546e964-9366-4c4e-9143-f83d51e00e43`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858921037824', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:55:58,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`d546e964-9366-4c4e-9143-f83d51e00e43`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:55:58,115::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:55:58,115::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:55:58,116::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:55:58,116::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:55:58,116::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:55:58,117::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:55:58,117::task::993::Storage.TaskManager.Task::(_decref) Task=`d546e964-9366-4c4e-9143-f83d51e00e43`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:55:58,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:55:58,216::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133241 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:00,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:00,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133206 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:02,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:02,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021878 s, 46.8 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:56:02,861::task::595::Storage.TaskManager.Task::(_updateState) Task=`76bfc42d-fc4f-40ef-ba7c-84f67271e0d6`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:02,862::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:56:02,887::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421955', 'lastCheck': '7.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:56:02,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`76bfc42d-fc4f-40ef-ba7c-84f67271e0d6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421955', 'lastCheck': '7.0', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:56:02,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`76bfc42d-fc4f-40ef-ba7c-84f67271e0d6`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:02,891::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:56:02,892::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:02,893::task::993::Storage.TaskManager.Task::(_decref) Task=`76bfc42d-fc4f-40ef-ba7c-84f67271e0d6`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:02,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:02,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:02,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:02,943::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:56:04,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:04,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132919 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:05,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:05,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:05,908::__init__::232::IOProcess::(_processLogs) (5997) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:56:05,909::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:56:05,928::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417624 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:05,929::__init__::232::IOProcess::(_processLogs) (5998) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:05,930::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:05,931::__init__::232::IOProcess::(_processLogs) (5999) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:05,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:05,932::__init__::232::IOProcess::(_processLogs) (6000) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:05,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:05,933::__init__::232::IOProcess::(_processLogs) (6001) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:05,934::__init__::232::IOProcess::(_processLogs) (6001) Queuing response >Dummy-33::DEBUG::2014-08-11 14:56:06,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:06,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131243 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:07,926::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:07,932::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:07,987::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:08,128::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:56:08,129::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c75b6c2-9da5-4af7-a035-a6783dad01e2`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:08,129::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:56:08,143::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:08,143::task::1191::Storage.TaskManager.Task::(prepare) Task=`7c75b6c2-9da5-4af7-a035-a6783dad01e2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:08,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c75b6c2-9da5-4af7-a035-a6783dad01e2`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:08,144::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:56:08,144::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:08,144::task::993::Storage.TaskManager.Task::(_decref) Task=`7c75b6c2-9da5-4af7-a035-a6783dad01e2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:56:08,158::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:56:08,159::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a61ca36-0f69-47fa-9c2c-353f4af380f8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:56:08,159::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:56:08,160::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9a4ea55-c228-4b1d-987c-27319895ddab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:56:08,160::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:56:08,160::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:56:08,160::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9a4ea55-c228-4b1d-987c-27319895ddab`::Granted request >Thread-34::DEBUG::2014-08-11 14:56:08,161::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4a61ca36-0f69-47fa-9c2c-353f4af380f8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:56:08,161::task::993::Storage.TaskManager.Task::(_decref) Task=`4a61ca36-0f69-47fa-9c2c-353f4af380f8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:56:08,181::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858764079104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:08,181::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a61ca36-0f69-47fa-9c2c-353f4af380f8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858764079104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:08,182::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a61ca36-0f69-47fa-9c2c-353f4af380f8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:56:08,182::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:56:08,182::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:56:08,182::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:56:08,183::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:56:08,183::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:56:08,183::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:56:08,183::task::993::Storage.TaskManager.Task::(_decref) Task=`4a61ca36-0f69-47fa-9c2c-353f4af380f8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:56:08,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:08,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131783 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:10,758::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:10,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131188 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:12,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:12,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133303 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:12,991::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:12,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:13,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:56:14,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`624dac9f-c19c-4380-8f7b-9f07d53baf2e`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:56:14,132::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:56:14,133::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:14,134::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:14,134::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:14,134::__init__::232::IOProcess::(_processLogs) (6002) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:56:14,135::__init__::232::IOProcess::(_processLogs) DEBUG|(6002) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:14,136::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:14,136::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:14,137::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:14,137::__init__::232::IOProcess::(_processLogs) (6003) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:56:14,137::__init__::232::IOProcess::(_processLogs) DEBUG|(6003) Queuing response >Thread-55::INFO::2014-08-11 14:56:14,138::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:56:14,139::task::1191::Storage.TaskManager.Task::(prepare) Task=`624dac9f-c19c-4380-8f7b-9f07d53baf2e`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:56:14,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`624dac9f-c19c-4380-8f7b-9f07d53baf2e`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:56:14,139::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:56:14,139::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:56:14,139::task::993::Storage.TaskManager.Task::(_decref) Task=`624dac9f-c19c-4380-8f7b-9f07d53baf2e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:56:14,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:14,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131676 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:15,935::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:15,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:15,936::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:15,936::__init__::232::IOProcess::(_processLogs) (6004) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:15,936::__init__::232::IOProcess::(_processLogs) (6004) Queuing response >Thread-16::DEBUG::2014-08-11 14:56:15,937::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:56:15,956::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000502376 s, 906 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:15,957::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:15,958::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:15,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:15,958::__init__::232::IOProcess::(_processLogs) (6005) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:15,959::__init__::232::IOProcess::(_processLogs) (6005) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:15,960::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:15,960::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:15,960::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:15,961::__init__::232::IOProcess::(_processLogs) DEBUG|(6006) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:15,961::__init__::232::IOProcess::(_processLogs) (6006) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:15,962::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:15,962::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:15,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:15,963::__init__::232::IOProcess::(_processLogs) DEBUG|(6007) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:15,963::__init__::232::IOProcess::(_processLogs) (6007) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:15,964::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:15,964::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:15,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:15,964::__init__::232::IOProcess::(_processLogs) DEBUG|(6008) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:15,965::__init__::232::IOProcess::(_processLogs) (6008) Queuing response >Dummy-33::DEBUG::2014-08-11 14:56:16,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:16,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01307 s, 78.3 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:56:18,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`005ece13-a06c-45e7-afe6-93ae67c52b05`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:18,044::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-103::INFO::2014-08-11 14:56:18,088::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000502376', 'lastCheck': '2.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:56:18,088::task::1191::Storage.TaskManager.Task::(prepare) Task=`005ece13-a06c-45e7-afe6-93ae67c52b05`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000502376', 'lastCheck': '2.1', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:56:18,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`005ece13-a06c-45e7-afe6-93ae67c52b05`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:18,088::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:56:18,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:18,089::task::993::Storage.TaskManager.Task::(_decref) Task=`005ece13-a06c-45e7-afe6-93ae67c52b05`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:18,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:18,097::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:18,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:18,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:18,196::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:56:18,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc15fd1f-7947-4ce8-825c-22e6e6b43ca4`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:18,197::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:56:18,209::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:18,210::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc15fd1f-7947-4ce8-825c-22e6e6b43ca4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:18,210::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc15fd1f-7947-4ce8-825c-22e6e6b43ca4`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:18,210::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:56:18,210::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:18,211::task::993::Storage.TaskManager.Task::(_decref) Task=`dc15fd1f-7947-4ce8-825c-22e6e6b43ca4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:56:18,221::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:56:18,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa2b7311-ca1d-4433-a455-b3da164905df`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:56:18,222::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:56:18,222::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`17de3a14-5243-44b1-b2b9-8ab5f1f76d5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:56:18,223::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:56:18,223::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:56:18,223::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`17de3a14-5243-44b1-b2b9-8ab5f1f76d5c`::Granted request >Thread-34::DEBUG::2014-08-11 14:56:18,224::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aa2b7311-ca1d-4433-a455-b3da164905df`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:56:18,224::task::993::Storage.TaskManager.Task::(_decref) Task=`aa2b7311-ca1d-4433-a455-b3da164905df`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:56:18,244::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:18,244::task::1191::Storage.TaskManager.Task::(prepare) Task=`aa2b7311-ca1d-4433-a455-b3da164905df`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:18,244::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa2b7311-ca1d-4433-a455-b3da164905df`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:56:18,245::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:56:18,245::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:56:18,245::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:56:18,245::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:56:18,246::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:56:18,246::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:56:18,246::task::993::Storage.TaskManager.Task::(_decref) Task=`aa2b7311-ca1d-4433-a455-b3da164905df`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:56:18,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:19,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132634 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:21,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:21,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133524 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:23,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:23,110::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:56:23,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133447 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:23,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:23,197::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:56:25,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:25,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132151 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:25,967::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:25,967::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:25,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:25,968::__init__::232::IOProcess::(_processLogs) (6009) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:25,968::__init__::232::IOProcess::(_processLogs) (6009) Queuing response >Thread-16::DEBUG::2014-08-11 14:56:25,969::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:56:25,987::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425551 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:25,988::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:25,989::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:25,989::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:25,989::__init__::232::IOProcess::(_processLogs) (6010) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:25,990::__init__::232::IOProcess::(_processLogs) DEBUG|(6010) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:25,991::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:25,992::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:25,993::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:25,993::__init__::232::IOProcess::(_processLogs) (6011) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:25,993::__init__::232::IOProcess::(_processLogs) (6011) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:25,994::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:25,994::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:25,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:25,994::__init__::232::IOProcess::(_processLogs) (6012) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:25,995::__init__::232::IOProcess::(_processLogs) (6012) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:25,996::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:25,996::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:25,997::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:25,998::__init__::232::IOProcess::(_processLogs) (6013) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:25,998::__init__::232::IOProcess::(_processLogs) (6013) Queuing response >Dummy-33::DEBUG::2014-08-11 14:56:27,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:27,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131362 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:28,207::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:28,208::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:28,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:28,260::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:56:28,261::task::595::Storage.TaskManager.Task::(_updateState) Task=`e10683f5-775d-44ed-90e9-17e0e690a7f2`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:28,261::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:56:28,274::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:28,274::task::1191::Storage.TaskManager.Task::(prepare) Task=`e10683f5-775d-44ed-90e9-17e0e690a7f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:28,274::task::595::Storage.TaskManager.Task::(_updateState) Task=`e10683f5-775d-44ed-90e9-17e0e690a7f2`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:28,275::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:56:28,275::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:28,275::task::993::Storage.TaskManager.Task::(_decref) Task=`e10683f5-775d-44ed-90e9-17e0e690a7f2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:56:28,286::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:56:28,287::task::595::Storage.TaskManager.Task::(_updateState) Task=`aba995b4-bd9b-4492-a402-6ac5a27573b8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:56:28,287::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:56:28,288::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c5a5bfb7-3e3f-40c3-85c3-f1d17aecf000`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:56:28,288::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:56:28,289::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:56:28,289::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c5a5bfb7-3e3f-40c3-85c3-f1d17aecf000`::Granted request >Thread-34::DEBUG::2014-08-11 14:56:28,291::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aba995b4-bd9b-4492-a402-6ac5a27573b8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:56:28,291::task::993::Storage.TaskManager.Task::(_decref) Task=`aba995b4-bd9b-4492-a402-6ac5a27573b8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:56:28,311::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:28,311::task::1191::Storage.TaskManager.Task::(prepare) Task=`aba995b4-bd9b-4492-a402-6ac5a27573b8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:28,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`aba995b4-bd9b-4492-a402-6ac5a27573b8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:56:28,311::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:56:28,312::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:56:28,312::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:56:28,312::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:56:28,313::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:56:28,313::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:56:28,313::task::993::Storage.TaskManager.Task::(_decref) Task=`aba995b4-bd9b-4492-a402-6ac5a27573b8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:56:29,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:29,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137947 s, 74.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:31,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:31,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133589 s, 76.7 MB/s\n'; <rc> = 0 >Thread-103::DEBUG::2014-08-11 14:56:33,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7e78cc5-1ba3-405d-9584-3ec45f129166`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:33,200::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:33,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::INFO::2014-08-11 14:56:33,289::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425551', 'lastCheck': '7.3', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:56:33,291::task::1191::Storage.TaskManager.Task::(prepare) Task=`f7e78cc5-1ba3-405d-9584-3ec45f129166`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425551', 'lastCheck': '7.3', 'valid': True}} >Thread-103::DEBUG::2014-08-11 14:56:33,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7e78cc5-1ba3-405d-9584-3ec45f129166`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:33,294::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:33,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:33,295::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:33,296::task::993::Storage.TaskManager.Task::(_decref) Task=`f7e78cc5-1ba3-405d-9584-3ec45f129166`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:33,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:33,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:56:33,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:33,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131981 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:35,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:35,531::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132381 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:35,998::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:35,999::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:36,000::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:56:36,001::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:56:36,001::__init__::232::IOProcess::(_processLogs) (6014) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:36,019::__init__::232::IOProcess::(_processLogs) (6014) Queuing response >Thread-16::DEBUG::2014-08-11 14:56:36,021::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000616787 s, 738 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:36,022::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:36,022::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:36,023::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:36,023::__init__::232::IOProcess::(_processLogs) (6015) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:36,024::__init__::232::IOProcess::(_processLogs) DEBUG|(6015) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:36,025::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:36,026::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:36,026::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:36,026::__init__::232::IOProcess::(_processLogs) (6016) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:36,027::__init__::232::IOProcess::(_processLogs) DEBUG|(6016) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:36,028::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:36,028::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:36,028::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:36,028::__init__::232::IOProcess::(_processLogs) (6017) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:36,029::__init__::232::IOProcess::(_processLogs) DEBUG|(6017) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:36,030::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:36,030::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:36,030::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:36,031::__init__::232::IOProcess::(_processLogs) (6018) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:36,031::__init__::232::IOProcess::(_processLogs) DEBUG|(6018) Queuing response >Dummy-33::DEBUG::2014-08-11 14:56:37,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:37,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132231 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:38,311::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:38,312::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:38,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:38,325::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:56:38,326::task::595::Storage.TaskManager.Task::(_updateState) Task=`b16ac727-c340-4f5b-a533-713f89e74c1f`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:38,326::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-103::INFO::2014-08-11 14:56:38,339::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:38,339::task::1191::Storage.TaskManager.Task::(prepare) Task=`b16ac727-c340-4f5b-a533-713f89e74c1f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:38,340::task::595::Storage.TaskManager.Task::(_updateState) Task=`b16ac727-c340-4f5b-a533-713f89e74c1f`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:38,340::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:56:38,340::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:38,340::task::993::Storage.TaskManager.Task::(_decref) Task=`b16ac727-c340-4f5b-a533-713f89e74c1f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:56:38,351::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:56:38,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`688f5b8c-cc31-4e68-ba68-382251636dac`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:56:38,352::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:56:38,352::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a88e7d22-b149-445a-b1e5-d7a38b10a053`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:56:38,353::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:56:38,353::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:56:38,353::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a88e7d22-b149-445a-b1e5-d7a38b10a053`::Granted request >Thread-34::DEBUG::2014-08-11 14:56:38,354::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`688f5b8c-cc31-4e68-ba68-382251636dac`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:56:38,354::task::993::Storage.TaskManager.Task::(_decref) Task=`688f5b8c-cc31-4e68-ba68-382251636dac`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:56:38,374::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:38,374::task::1191::Storage.TaskManager.Task::(prepare) Task=`688f5b8c-cc31-4e68-ba68-382251636dac`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:38,374::task::595::Storage.TaskManager.Task::(_updateState) Task=`688f5b8c-cc31-4e68-ba68-382251636dac`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:56:38,375::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:56:38,375::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:56:38,375::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:56:38,375::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:56:38,376::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:56:38,376::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:56:38,376::task::993::Storage.TaskManager.Task::(_decref) Task=`688f5b8c-cc31-4e68-ba68-382251636dac`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:56:39,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:39,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132398 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:41,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:41,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013169 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:43,320::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:43,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:43,322::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:56:43,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:43,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212331 s, 48.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:45,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:45,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131792 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:46,033::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:46,033::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:46,033::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:46,034::__init__::232::IOProcess::(_processLogs) DEBUG|(6019) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:56:46,036::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:56:46,036::__init__::232::IOProcess::(_processLogs) (6019) Queuing response >Thread-16::DEBUG::2014-08-11 14:56:46,055::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419537 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:46,056::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:56:46,057::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:46,057::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:46,057::__init__::232::IOProcess::(_processLogs) (6020) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:46,059::__init__::232::IOProcess::(_processLogs) (6021) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:46,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:46,060::__init__::232::IOProcess::(_processLogs) (6022) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:46,060::__init__::232::IOProcess::(_processLogs) (6022) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:46,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:46,061::__init__::232::IOProcess::(_processLogs) (6023) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:46,061::__init__::232::IOProcess::(_processLogs) (6023) Queuing response >Dummy-33::DEBUG::2014-08-11 14:56:47,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:47,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135259 s, 75.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:48,331::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:48,338::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::DEBUG::2014-08-11 14:56:48,388::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-103::DEBUG::2014-08-11 14:56:48,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`e96b58a3-b8a3-485e-b795-3ee2c39b5c42`::moving from state init -> state preparing >Thread-103::INFO::2014-08-11 14:56:48,392::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:48,393::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-103::INFO::2014-08-11 14:56:48,406::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:48,407::task::1191::Storage.TaskManager.Task::(prepare) Task=`e96b58a3-b8a3-485e-b795-3ee2c39b5c42`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-103::DEBUG::2014-08-11 14:56:48,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`e96b58a3-b8a3-485e-b795-3ee2c39b5c42`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-08-11 14:56:48,407::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-08-11 14:56:48,408::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-08-11 14:56:48,408::task::993::Storage.TaskManager.Task::(_decref) Task=`e96b58a3-b8a3-485e-b795-3ee2c39b5c42`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:56:48,418::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:56:48,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`270bc2b0-5d05-4e20-aef5-112f0b7e0f42`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:56:48,419::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:56:48,419::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f893793b-50e8-4184-b6c1-1d796a94925a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:56:48,420::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:56:48,420::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:56:48,420::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f893793b-50e8-4184-b6c1-1d796a94925a`::Granted request >Thread-34::DEBUG::2014-08-11 14:56:48,421::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`270bc2b0-5d05-4e20-aef5-112f0b7e0f42`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:56:48,421::task::993::Storage.TaskManager.Task::(_decref) Task=`270bc2b0-5d05-4e20-aef5-112f0b7e0f42`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:56:48,441::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:48,441::task::1191::Storage.TaskManager.Task::(prepare) Task=`270bc2b0-5d05-4e20-aef5-112f0b7e0f42`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738454528', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:48,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`270bc2b0-5d05-4e20-aef5-112f0b7e0f42`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:56:48,442::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:56:48,442::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:56:48,442::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:56:48,443::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:56:48,443::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:56:48,443::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:56:48,443::task::993::Storage.TaskManager.Task::(_decref) Task=`270bc2b0-5d05-4e20-aef5-112f0b7e0f42`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 14:56:48,594::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:54633 >Detector thread::DEBUG::2014-08-11 14:56:48,603::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:54633 >Detector thread::DEBUG::2014-08-11 14:56:48,604::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:54633 >Detector thread::DEBUG::2014-08-11 14:56:48,604::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 54633) >Thread-106::DEBUG::2014-08-11 14:56:48,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb99b72-8d31-4a8e-9980-ea9290441927`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:56:48,608::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:56:48,608::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419537', 'lastCheck': '2.5', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:56:48,608::task::1191::Storage.TaskManager.Task::(prepare) Task=`beb99b72-8d31-4a8e-9980-ea9290441927`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419537', 'lastCheck': '2.5', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:56:48,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`beb99b72-8d31-4a8e-9980-ea9290441927`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:56:48,609::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:56:48,609::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:56:48,609::task::993::Storage.TaskManager.Task::(_decref) Task=`beb99b72-8d31-4a8e-9980-ea9290441927`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:56:48,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:56:49,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:49,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132713 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:56:51,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:51,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132879 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:53,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:53,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:53,490::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:56:54,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:54,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131428 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:56,063::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:56,064::__init__::232::IOProcess::(_processLogs) (6024) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:56:56,064::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:56,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 14:56:56,108::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00397379 s, 115 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:56,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:56,110::__init__::232::IOProcess::(_processLogs) (6025) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:56:56,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:56,123::__init__::232::IOProcess::(_processLogs) (6026) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:56,124::__init__::232::IOProcess::(_processLogs) (6026) Queuing response >Thread-14::DEBUG::2014-08-11 14:56:56,126::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:56:56,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:56,128::__init__::232::IOProcess::(_processLogs) (6027) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:56:56,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0256229 s, 40.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:56:56,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:56:56,140::__init__::232::IOProcess::(_processLogs) (6028) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:56:56,141::__init__::232::IOProcess::(_processLogs) (6028) Queuing response >Dummy-33::DEBUG::2014-08-11 14:56:58,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:56:58,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132493 s, 77.3 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:56:58,459::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:56:58,462::task::595::Storage.TaskManager.Task::(_updateState) Task=`abc93db0-08e5-4308-ad1a-3bba642a6843`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:56:58,490::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:58,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:58,500::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:56:58,502::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::INFO::2014-08-11 14:56:58,504::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:56:58,505::task::1191::Storage.TaskManager.Task::(prepare) Task=`abc93db0-08e5-4308-ad1a-3bba642a6843`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:56:58,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`abc93db0-08e5-4308-ad1a-3bba642a6843`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:56:58,505::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:56:58,505::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:56:58,505::task::993::Storage.TaskManager.Task::(_decref) Task=`abc93db0-08e5-4308-ad1a-3bba642a6843`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:56:58,515::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:56:58,516::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b0a43ef-717b-40d9-9a14-4c1df0344a77`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:56:58,516::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:56:58,517::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c307a073-2c9e-4d98-975c-7896c0368bdf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:56:58,517::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:56:58,518::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:56:58,518::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c307a073-2c9e-4d98-975c-7896c0368bdf`::Granted request >Thread-34::DEBUG::2014-08-11 14:56:58,518::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6b0a43ef-717b-40d9-9a14-4c1df0344a77`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:56:58,519::task::993::Storage.TaskManager.Task::(_decref) Task=`6b0a43ef-717b-40d9-9a14-4c1df0344a77`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:56:58,539::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:58,539::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b0a43ef-717b-40d9-9a14-4c1df0344a77`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:56:58,539::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b0a43ef-717b-40d9-9a14-4c1df0344a77`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:56:58,539::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:56:58,540::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:56:58,540::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:56:58,540::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:56:58,540::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:56:58,541::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:56:58,541::task::993::Storage.TaskManager.Task::(_decref) Task=`6b0a43ef-717b-40d9-9a14-4c1df0344a77`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:57:00,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:00,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133642 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:02,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:02,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0311197 s, 32.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:03,508::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:03,510::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:03,591::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:03,739::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e34d802-4550-4f60-85c3-a8cdd3899ffd`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:03,739::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:57:03,740::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00397379', 'lastCheck': '7.6', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:03,740::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e34d802-4550-4f60-85c3-a8cdd3899ffd`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00397379', 'lastCheck': '7.6', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:03,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e34d802-4550-4f60-85c3-a8cdd3899ffd`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:03,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:03,741::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:03,741::task::993::Storage.TaskManager.Task::(_decref) Task=`5e34d802-4550-4f60-85c3-a8cdd3899ffd`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:57:03,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:57:04,575::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:04,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132009 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:06,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:06,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:06,141::__init__::232::IOProcess::(_processLogs) (6029) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:06,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6029) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:06,143::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:57:06,161::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000431866 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:06,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:06,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:06,163::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:06,164::__init__::232::IOProcess::(_processLogs) (6030) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:06,164::__init__::232::IOProcess::(_processLogs) (6030) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:06,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:06,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:06,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:06,166::__init__::232::IOProcess::(_processLogs) (6031) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:06,167::__init__::232::IOProcess::(_processLogs) (6031) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:06,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:06,168::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:06,168::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:06,169::__init__::232::IOProcess::(_processLogs) (6032) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:06,169::__init__::232::IOProcess::(_processLogs) (6032) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:06,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:06,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:06,170::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:06,171::__init__::232::IOProcess::(_processLogs) (6033) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:06,172::__init__::232::IOProcess::(_processLogs) (6033) Queuing response >Dummy-33::DEBUG::2014-08-11 14:57:06,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:06,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131666 s, 77.8 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:57:08,555::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:57:08,558::task::595::Storage.TaskManager.Task::(_updateState) Task=`39b2ea6c-a538-4f33-92d5-6ff9af73c29b`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:08,591::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:08,597::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:08,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:08,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::INFO::2014-08-11 14:57:08,605::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:08,606::task::1191::Storage.TaskManager.Task::(prepare) Task=`39b2ea6c-a538-4f33-92d5-6ff9af73c29b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:08,606::task::595::Storage.TaskManager.Task::(_updateState) Task=`39b2ea6c-a538-4f33-92d5-6ff9af73c29b`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:08,606::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:08,606::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:08,606::task::993::Storage.TaskManager.Task::(_decref) Task=`39b2ea6c-a538-4f33-92d5-6ff9af73c29b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:57:08,617::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:57:08,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b3981b0-71fb-4a03-9548-613d117107d6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:57:08,618::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:57:08,619::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5ed4c9c5-7c1f-4b7a-a9a1-94a1abbaaae0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:57:08,619::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:57:08,620::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:57:08,620::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5ed4c9c5-7c1f-4b7a-a9a1-94a1abbaaae0`::Granted request >Thread-34::DEBUG::2014-08-11 14:57:08,621::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3b3981b0-71fb-4a03-9548-613d117107d6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:57:08,621::task::993::Storage.TaskManager.Task::(_decref) Task=`3b3981b0-71fb-4a03-9548-613d117107d6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:57:08,641::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:08,641::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b3981b0-71fb-4a03-9548-613d117107d6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:08,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b3981b0-71fb-4a03-9548-613d117107d6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:57:08,641::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:57:08,642::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:57:08,642::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:57:08,642::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:57:08,643::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:57:08,643::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:57:08,643::task::993::Storage.TaskManager.Task::(_decref) Task=`3b3981b0-71fb-4a03-9548-613d117107d6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:57:08,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:08,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131824 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:10,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:10,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132424 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:12,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:12,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131869 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:13,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:13,609::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:13,610::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:57:14,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`fefb144d-001b-4c1f-9540-c73b78ed11bf`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:57:14,246::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:57:14,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:14,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:14,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:14,248::__init__::232::IOProcess::(_processLogs) (6034) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:57:14,249::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6034) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:14,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:14,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:57:14,251::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:57:14,252::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:57:14,252::task::1191::Storage.TaskManager.Task::(prepare) Task=`fefb144d-001b-4c1f-9540-c73b78ed11bf`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:57:14,253::__init__::232::IOProcess::(_processLogs) (6035) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:57:14,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`fefb144d-001b-4c1f-9540-c73b78ed11bf`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:57:14,255::__init__::232::IOProcess::(_processLogs) (6035) Queuing response >Thread-55::DEBUG::2014-08-11 14:57:14,255::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:57:14,256::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:57:14,256::task::993::Storage.TaskManager.Task::(_decref) Task=`fefb144d-001b-4c1f-9540-c73b78ed11bf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:57:14,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:14,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013217 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:16,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:16,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:16,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 14:57:16,174::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:57:16,175::__init__::232::IOProcess::(_processLogs) (6036) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:16,193::__init__::232::IOProcess::(_processLogs) (6036) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:16,195::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000465211 s, 978 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:16,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:16,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:16,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:16,197::__init__::232::IOProcess::(_processLogs) (6037) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:16,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6037) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:16,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:16,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:16,199::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:16,200::__init__::232::IOProcess::(_processLogs) (6038) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:16,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6038) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:16,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:16,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:16,202::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:16,202::__init__::232::IOProcess::(_processLogs) (6039) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:16,203::__init__::232::IOProcess::(_processLogs) (6039) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:16,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:16,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:16,204::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:16,204::__init__::232::IOProcess::(_processLogs) (6040) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:16,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6040) Queuing response >Dummy-33::DEBUG::2014-08-11 14:57:16,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:16,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131575 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:18,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:18,643::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:18,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:18,660::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:57:18,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`e768d52d-ccd3-4a4e-8c64-484588f56391`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:18,667::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:57:18,710::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:18,713::task::1191::Storage.TaskManager.Task::(prepare) Task=`e768d52d-ccd3-4a4e-8c64-484588f56391`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:18,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`e768d52d-ccd3-4a4e-8c64-484588f56391`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:18,715::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:18,716::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:18,718::task::993::Storage.TaskManager.Task::(_decref) Task=`e768d52d-ccd3-4a4e-8c64-484588f56391`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:57:18,730::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:57:18,732::task::595::Storage.TaskManager.Task::(_updateState) Task=`b559aaf1-c2d4-436b-a539-930465117aaa`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:57:18,734::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:57:18,737::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0e6cbb6-2f03-4fcd-ae31-287b1365589a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:57:18,738::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:57:18,740::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:57:18,741::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0e6cbb6-2f03-4fcd-ae31-287b1365589a`::Granted request >Thread-34::DEBUG::2014-08-11 14:57:18,744::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b559aaf1-c2d4-436b-a539-930465117aaa`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:57:18,745::task::993::Storage.TaskManager.Task::(_decref) Task=`b559aaf1-c2d4-436b-a539-930465117aaa`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:57:18,794::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:18,795::task::1191::Storage.TaskManager.Task::(prepare) Task=`b559aaf1-c2d4-436b-a539-930465117aaa`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:18,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`b559aaf1-c2d4-436b-a539-930465117aaa`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:57:18,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:57:18,798::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:57:18,800::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:57:18,802::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:57:18,803::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:57:18,804::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:57:18,806::task::993::Storage.TaskManager.Task::(_decref) Task=`b559aaf1-c2d4-436b-a539-930465117aaa`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:57:18,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9ecda66-2f01-4057-8190-4bbe532c6cac`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:18,867::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:57:18,867::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000465211', 'lastCheck': '2.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:18,868::task::1191::Storage.TaskManager.Task::(prepare) Task=`e9ecda66-2f01-4057-8190-4bbe532c6cac`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000465211', 'lastCheck': '2.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:18,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`e9ecda66-2f01-4057-8190-4bbe532c6cac`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:18,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:18,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:18,868::task::993::Storage.TaskManager.Task::(_decref) Task=`e9ecda66-2f01-4057-8190-4bbe532c6cac`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:57:18,898::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:57:18,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:18,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132877 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:21,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:21,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132399 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:23,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:23,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013176 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:23,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:23,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:23,664::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:57:25,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:25,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132157 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:26,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:26,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:26,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:26,292::__init__::232::IOProcess::(_processLogs) (6041) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:57:26,292::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:57:26,294::__init__::232::IOProcess::(_processLogs) (6041) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:26,313::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00049668 s, 916 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:26,314::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:26,314::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:26,314::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:26,315::__init__::232::IOProcess::(_processLogs) (6042) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:26,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6042) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:26,316::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:26,316::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:26,317::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:26,317::__init__::232::IOProcess::(_processLogs) (6043) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:26,317::__init__::232::IOProcess::(_processLogs) (6043) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:26,318::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:26,318::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:26,319::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:26,319::__init__::232::IOProcess::(_processLogs) (6044) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:26,319::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6044) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:26,320::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:26,321::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:26,321::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:26,321::__init__::232::IOProcess::(_processLogs) (6045) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:26,322::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|(6045) Queuing response >Dummy-33::DEBUG::2014-08-11 14:57:27,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:27,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131948 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:28,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:28,669::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:28,670::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:28,823::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-106::DEBUG::2014-08-11 14:57:28,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`aabf3373-593f-4bc1-80c3-95ccb7dc42a6`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:28,826::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:57:28,879::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:28,881::task::1191::Storage.TaskManager.Task::(prepare) Task=`aabf3373-593f-4bc1-80c3-95ccb7dc42a6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:28,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`aabf3373-593f-4bc1-80c3-95ccb7dc42a6`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:28,884::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:28,885::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:28,887::task::993::Storage.TaskManager.Task::(_decref) Task=`aabf3373-593f-4bc1-80c3-95ccb7dc42a6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:57:28,904::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 14:57:28,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`6306e5e6-49e5-43b7-ad26-c5bda0c625f8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:57:28,909::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:57:28,912::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b98306fc-eed9-4270-bb58-cc914705926e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:57:28,913::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:57:28,915::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:57:28,916::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b98306fc-eed9-4270-bb58-cc914705926e`::Granted request >Thread-34::DEBUG::2014-08-11 14:57:28,918::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6306e5e6-49e5-43b7-ad26-c5bda0c625f8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:57:28,920::task::993::Storage.TaskManager.Task::(_decref) Task=`6306e5e6-49e5-43b7-ad26-c5bda0c625f8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:57:28,966::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:28,968::task::1191::Storage.TaskManager.Task::(prepare) Task=`6306e5e6-49e5-43b7-ad26-c5bda0c625f8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:28,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`6306e5e6-49e5-43b7-ad26-c5bda0c625f8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:57:28,970::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:57:28,971::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:57:28,973::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:57:28,976::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:57:28,977::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:57:28,978::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:57:28,980::task::993::Storage.TaskManager.Task::(_decref) Task=`6306e5e6-49e5-43b7-ad26-c5bda0c625f8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:57:29,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:29,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133254 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:31,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:31,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133447 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:33,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:33,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132252 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:33,677::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:33,678::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:33,679::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:33,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`af1a780c-f986-490c-835a-36dcb1d61608`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:33,989::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:57:33,990::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00049668', 'lastCheck': '7.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:33,990::task::1191::Storage.TaskManager.Task::(prepare) Task=`af1a780c-f986-490c-835a-36dcb1d61608`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00049668', 'lastCheck': '7.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:33,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`af1a780c-f986-490c-835a-36dcb1d61608`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:33,990::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:33,991::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:33,991::task::993::Storage.TaskManager.Task::(_decref) Task=`af1a780c-f986-490c-835a-36dcb1d61608`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:57:34,010::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:57:35,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:35,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131403 s, 77.9 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 14:57:36,323::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:57:36,324::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,325::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,325::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,326::__init__::232::IOProcess::(_processLogs) (6046) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:57:36,326::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|(6046) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:36,334::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 14:57:36,335::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,391::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:57:36,391::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 14:57:36,392::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,393::__init__::232::IOProcess::(_processLogs) (6047) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 14:57:36,394::__init__::232::IOProcess::(_processLogs) (6047) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:36,394::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,394::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,395::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,395::__init__::232::IOProcess::(_processLogs) (6048) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 14:57:36,395::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|(6048) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:36,396::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 14:57:36,397::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,397::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,397::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,398::__init__::232::IOProcess::(_processLogs) (6049) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:36,398::__init__::232::IOProcess::(_processLogs) (6049) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:36,399::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,399::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,399::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,400::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|(6050) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 14:57:36,400::__init__::232::IOProcess::(_processLogs) (6050) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:36,400::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 14:57:36,400::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 14:57:36,401::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 14:57:36,401::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,402::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,402::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,402::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|(6051) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:36,402::__init__::232::IOProcess::(_processLogs) (6051) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:36,403::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:57:36,422::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416942 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:36,423::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,424::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,424::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,424::__init__::232::IOProcess::(_processLogs) (6052) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:36,424::__init__::232::IOProcess::(_processLogs) (6052) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:36,425::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,426::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,426::__init__::232::IOProcess::(_processLogs) (6053) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:36,427::__init__::232::IOProcess::(_processLogs) (6053) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:36,427::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,428::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,428::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,428::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6054) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:36,428::__init__::232::IOProcess::(_processLogs) (6054) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:36,429::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:36,430::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:36,430::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:36,430::__init__::232::IOProcess::(_processLogs) (6055) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:36,430::__init__::232::IOProcess::(_processLogs) (6055) Queuing response >Dummy-33::DEBUG::2014-08-11 14:57:37,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:37,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132892 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:38,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:38,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:38,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:39,000::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:57:39,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`19718f7b-ebdb-438b-8f36-ea9c059439af`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:39,003::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:57:39,031::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:39,033::task::1191::Storage.TaskManager.Task::(prepare) Task=`19718f7b-ebdb-438b-8f36-ea9c059439af`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:39,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`19718f7b-ebdb-438b-8f36-ea9c059439af`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:39,035::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:39,036::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:39,037::task::993::Storage.TaskManager.Task::(_decref) Task=`19718f7b-ebdb-438b-8f36-ea9c059439af`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:57:39,052::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:57:39,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`20520aab-fc51-4e1a-a2c9-03140354b055`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:57:39,057::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:57:39,062::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a5aac90f-f994-4524-9fa9-9ff4495baaf4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:57:39,063::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:57:39,065::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:57:39,067::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a5aac90f-f994-4524-9fa9-9ff4495baaf4`::Granted request >Thread-34::DEBUG::2014-08-11 14:57:39,071::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`20520aab-fc51-4e1a-a2c9-03140354b055`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:57:39,072::task::993::Storage.TaskManager.Task::(_decref) Task=`20520aab-fc51-4e1a-a2c9-03140354b055`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:57:39,133::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:39,135::task::1191::Storage.TaskManager.Task::(prepare) Task=`20520aab-fc51-4e1a-a2c9-03140354b055`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:39,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`20520aab-fc51-4e1a-a2c9-03140354b055`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:57:39,137::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:57:39,138::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:57:39,140::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:57:39,142::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:57:39,144::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:57:39,145::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:57:39,148::task::993::Storage.TaskManager.Task::(_decref) Task=`20520aab-fc51-4e1a-a2c9-03140354b055`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:57:39,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:39,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131973 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:41,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:41,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013285 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:43,689::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:43,690::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:43,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:57:43,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:43,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131443 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:45,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:45,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132955 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:46,432::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:46,433::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:46,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:46,433::__init__::232::IOProcess::(_processLogs) (6056) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:46,433::__init__::232::IOProcess::(_processLogs) (6056) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:46,434::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:57:46,453::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0004205 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:46,454::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:46,455::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:46,455::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:46,455::__init__::232::IOProcess::(_processLogs) (6057) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:46,455::__init__::232::IOProcess::(_processLogs) (6057) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:46,456::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:46,457::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:46,458::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:46,458::__init__::232::IOProcess::(_processLogs) (6058) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:46,458::__init__::232::IOProcess::(_processLogs) (6058) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:46,458::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:46,459::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:46,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:46,460::__init__::232::IOProcess::(_processLogs) (6059) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:46,460::__init__::232::IOProcess::(_processLogs) (6059) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:46,461::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:46,461::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:46,462::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:46,462::__init__::232::IOProcess::(_processLogs) (6060) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:46,463::__init__::232::IOProcess::(_processLogs) (6060) Queuing response >Dummy-33::DEBUG::2014-08-11 14:57:47,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:47,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134788 s, 76.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:48,696::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:48,698::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:48,698::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:49,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`615c6983-704b-4376-8172-2ef61e93f1bb`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:49,114::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:57:49,114::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0004205', 'lastCheck': '2.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:49,115::task::1191::Storage.TaskManager.Task::(prepare) Task=`615c6983-704b-4376-8172-2ef61e93f1bb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0004205', 'lastCheck': '2.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:57:49,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`615c6983-704b-4376-8172-2ef61e93f1bb`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:49,115::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:49,115::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:49,116::task::993::Storage.TaskManager.Task::(_decref) Task=`615c6983-704b-4376-8172-2ef61e93f1bb`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:57:49,145::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:49,164::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:57:49,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`571f4aea-cdc5-42b5-881f-455c665a1d42`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:49,165::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:57:49,178::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:49,178::task::1191::Storage.TaskManager.Task::(prepare) Task=`571f4aea-cdc5-42b5-881f-455c665a1d42`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:49,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`571f4aea-cdc5-42b5-881f-455c665a1d42`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:49,178::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:49,179::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:49,179::task::993::Storage.TaskManager.Task::(_decref) Task=`571f4aea-cdc5-42b5-881f-455c665a1d42`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:57:49,191::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:57:49,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1846ed7-1706-4b19-aa05-72c8b9157f5e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:57:49,194::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:57:49,200::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0daad04c-fd8e-40ea-b7f3-54ae178adf2d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:57:49,201::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:57:49,202::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:57:49,203::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0daad04c-fd8e-40ea-b7f3-54ae178adf2d`::Granted request >Thread-34::DEBUG::2014-08-11 14:57:49,206::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d1846ed7-1706-4b19-aa05-72c8b9157f5e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:57:49,207::task::993::Storage.TaskManager.Task::(_decref) Task=`d1846ed7-1706-4b19-aa05-72c8b9157f5e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:57:49,256::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:49,257::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1846ed7-1706-4b19-aa05-72c8b9157f5e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738388992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:49,258::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1846ed7-1706-4b19-aa05-72c8b9157f5e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:57:49,260::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:57:49,261::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:57:49,263::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:57:49,264::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:57:49,265::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:57:49,266::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:57:49,268::task::993::Storage.TaskManager.Task::(_decref) Task=`d1846ed7-1706-4b19-aa05-72c8b9157f5e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:57:49,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:49,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133149 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:51,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:51,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131502 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:53,705::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:53,707::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:53,791::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:57:53,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:54,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131894 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:57:56,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:56,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132148 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:56,463::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:56,464::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:56,464::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:56,464::__init__::232::IOProcess::(_processLogs) (6061) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:56,465::__init__::232::IOProcess::(_processLogs) (6061) Queuing response >Thread-16::DEBUG::2014-08-11 14:57:56,465::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:57:56,484::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000449205 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:57:56,486::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:56,486::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:56,486::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:56,486::__init__::232::IOProcess::(_processLogs) (6062) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:57:56,487::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6062) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:56,488::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:56,489::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:56,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:56,490::__init__::232::IOProcess::(_processLogs) (6063) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:56,490::__init__::232::IOProcess::(_processLogs) (6063) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:56,491::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:56,491::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:56,492::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:56,492::__init__::232::IOProcess::(_processLogs) (6064) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:56,492::__init__::232::IOProcess::(_processLogs) (6064) Queuing response >Thread-14::DEBUG::2014-08-11 14:57:56,493::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:57:56,494::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:57:56,494::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:57:56,495::__init__::232::IOProcess::(_processLogs) (6065) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:57:56,495::__init__::232::IOProcess::(_processLogs) (6065) Queuing response >Dummy-33::DEBUG::2014-08-11 14:57:58,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:57:58,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132062 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:58,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:58,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:57:58,892::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:57:59,284::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:57:59,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fb3afc8-3bef-4cb3-996f-b88252870e5b`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:57:59,287::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:57:59,307::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:59,307::task::1191::Storage.TaskManager.Task::(prepare) Task=`4fb3afc8-3bef-4cb3-996f-b88252870e5b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:57:59,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fb3afc8-3bef-4cb3-996f-b88252870e5b`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:57:59,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:57:59,308::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:57:59,308::task::993::Storage.TaskManager.Task::(_decref) Task=`4fb3afc8-3bef-4cb3-996f-b88252870e5b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:57:59,318::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:57:59,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`458ca779-dafc-48c7-842f-14f42354ce0b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:57:59,320::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:57:59,320::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`61d50a46-0d6e-45d9-ba24-ec6e539b7add`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:57:59,321::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:57:59,321::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:57:59,321::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`61d50a46-0d6e-45d9-ba24-ec6e539b7add`::Granted request >Thread-34::DEBUG::2014-08-11 14:57:59,322::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`458ca779-dafc-48c7-842f-14f42354ce0b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:57:59,322::task::993::Storage.TaskManager.Task::(_decref) Task=`458ca779-dafc-48c7-842f-14f42354ce0b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:57:59,350::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738061312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:59,350::task::1191::Storage.TaskManager.Task::(prepare) Task=`458ca779-dafc-48c7-842f-14f42354ce0b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738061312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:57:59,351::task::595::Storage.TaskManager.Task::(_updateState) Task=`458ca779-dafc-48c7-842f-14f42354ce0b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:57:59,353::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:57:59,354::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:57:59,357::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:57:59,359::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:57:59,360::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:57:59,361::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:57:59,362::task::993::Storage.TaskManager.Task::(_decref) Task=`458ca779-dafc-48c7-842f-14f42354ce0b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:58:00,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:00,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133367 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:02,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:02,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304773 s, 33.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:03,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:03,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:03,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:04,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`161f336c-3a88-4549-8fdf-92d9d14f7b10`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:04,254::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:58:04,254::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449205', 'lastCheck': '7.8', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:04,254::task::1191::Storage.TaskManager.Task::(prepare) Task=`161f336c-3a88-4549-8fdf-92d9d14f7b10`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000449205', 'lastCheck': '7.8', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:04,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`161f336c-3a88-4549-8fdf-92d9d14f7b10`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:04,255::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:04,255::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:04,255::task::993::Storage.TaskManager.Task::(_decref) Task=`161f336c-3a88-4549-8fdf-92d9d14f7b10`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:58:04,275::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:58:04,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:04,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131791 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:06,495::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:06,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:06,497::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:06,498::__init__::232::IOProcess::(_processLogs) (6066) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:58:06,498::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:58:06,499::__init__::232::IOProcess::(_processLogs) (6066) Queuing response >Thread-16::DEBUG::2014-08-11 14:58:06,518::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000433539 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:06,519::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:06,520::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:06,520::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:06,520::__init__::232::IOProcess::(_processLogs) (6067) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:06,521::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6067) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:06,522::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:06,523::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:06,524::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:06,524::__init__::232::IOProcess::(_processLogs) (6068) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:06,524::__init__::232::IOProcess::(_processLogs) (6068) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:06,525::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:06,525::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:06,526::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:06,526::__init__::232::IOProcess::(_processLogs) (6069) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:06,526::__init__::232::IOProcess::(_processLogs) (6069) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:06,527::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:06,527::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:06,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:06,529::__init__::232::IOProcess::(_processLogs) (6070) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:06,529::__init__::232::IOProcess::(_processLogs) (6070) Queuing response >Dummy-33::DEBUG::2014-08-11 14:58:06,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:06,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013203 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:08,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:08,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132104 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:08,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:09,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:09,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:09,389::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:58:09,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c73e886-3029-45e1-9c92-f70375dccbac`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:09,392::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:58:09,410::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:09,410::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c73e886-3029-45e1-9c92-f70375dccbac`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:09,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c73e886-3029-45e1-9c92-f70375dccbac`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:09,411::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:09,411::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:09,411::task::993::Storage.TaskManager.Task::(_decref) Task=`5c73e886-3029-45e1-9c92-f70375dccbac`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:58:09,431::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:58:09,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`f13c28a1-cfce-4776-b93b-2b14ebff00f3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:58:09,434::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:58:09,436::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9663968f-195f-404c-b8d4-2336a2e70a21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:58:09,437::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:58:09,437::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:58:09,438::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9663968f-195f-404c-b8d4-2336a2e70a21`::Granted request >Thread-34::DEBUG::2014-08-11 14:58:09,442::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f13c28a1-cfce-4776-b93b-2b14ebff00f3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:58:09,452::task::993::Storage.TaskManager.Task::(_decref) Task=`f13c28a1-cfce-4776-b93b-2b14ebff00f3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:58:09,513::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738061312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:09,514::task::1191::Storage.TaskManager.Task::(prepare) Task=`f13c28a1-cfce-4776-b93b-2b14ebff00f3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858738061312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:09,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`f13c28a1-cfce-4776-b93b-2b14ebff00f3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:58:09,516::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:58:09,518::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:58:09,520::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:58:09,522::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:58:09,523::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:58:09,524::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:58:09,527::task::993::Storage.TaskManager.Task::(_decref) Task=`f13c28a1-cfce-4776-b93b-2b14ebff00f3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:58:10,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:10,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132236 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:12,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:12,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132302 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:14,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:14,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:14,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 14:58:14,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbd0ddf6-cf65-408b-a9b5-0ef9de75af8e`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:58:14,398::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:58:14,400::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:14,401::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:14,401::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:14,401::__init__::232::IOProcess::(_processLogs) (6071) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:58:14,402::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6071) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:14,403::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:14,404::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 14:58:14,403::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:58:14,405::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 14:58:14,405::task::1191::Storage.TaskManager.Task::(prepare) Task=`fbd0ddf6-cf65-408b-a9b5-0ef9de75af8e`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 14:58:14,406::__init__::232::IOProcess::(_processLogs) (6072) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 14:58:14,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`fbd0ddf6-cf65-408b-a9b5-0ef9de75af8e`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 14:58:14,407::__init__::232::IOProcess::(_processLogs) (6072) Queuing response >Thread-55::DEBUG::2014-08-11 14:58:14,408::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:58:14,409::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:58:14,409::task::993::Storage.TaskManager.Task::(_decref) Task=`fbd0ddf6-cf65-408b-a9b5-0ef9de75af8e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:58:14,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:14,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132859 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:16,530::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:16,530::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:16,530::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:16,531::__init__::232::IOProcess::(_processLogs) (6073) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:16,531::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6073) Queuing response >Thread-16::DEBUG::2014-08-11 14:58:16,533::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:58:16,551::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000413042 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:16,552::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:16,553::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:16,553::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:16,553::__init__::232::IOProcess::(_processLogs) (6074) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:16,554::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:16,555::__init__::232::IOProcess::(_processLogs) (6075) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:16,556::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:16,556::__init__::232::IOProcess::(_processLogs) (6076) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:16,556::__init__::232::IOProcess::(_processLogs) DEBUG|(6076) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:16,557::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:16,558::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:16,558::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:16,558::__init__::232::IOProcess::(_processLogs) (6077) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:16,559::__init__::232::IOProcess::(_processLogs) DEBUG|(6077) Queuing response >Dummy-33::DEBUG::2014-08-11 14:58:16,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:16,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131568 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:18,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:18,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131537 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:19,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:19,201::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:19,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:19,371::task::595::Storage.TaskManager.Task::(_updateState) Task=`8da2bb5b-fb92-4e1a-b142-81abb1318c5a`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:19,371::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:58:19,372::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413042', 'lastCheck': '2.8', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:19,372::task::1191::Storage.TaskManager.Task::(prepare) Task=`8da2bb5b-fb92-4e1a-b142-81abb1318c5a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000413042', 'lastCheck': '2.8', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:19,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`8da2bb5b-fb92-4e1a-b142-81abb1318c5a`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:19,372::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:19,372::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:19,373::task::993::Storage.TaskManager.Task::(_decref) Task=`8da2bb5b-fb92-4e1a-b142-81abb1318c5a`::ref 0 aborting False >Thread-106::DEBUG::2014-08-11 14:58:19,392::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:19,547::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:58:19,549::task::595::Storage.TaskManager.Task::(_updateState) Task=`e613813d-dc00-427e-a2b8-d4881c577395`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:19,549::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:58:19,572::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:19,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`e613813d-dc00-427e-a2b8-d4881c577395`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:19,575::task::595::Storage.TaskManager.Task::(_updateState) Task=`e613813d-dc00-427e-a2b8-d4881c577395`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:19,575::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:19,576::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:19,576::task::993::Storage.TaskManager.Task::(_decref) Task=`e613813d-dc00-427e-a2b8-d4881c577395`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:58:19,590::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:58:19,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`be53a31b-f11d-4503-9521-e620f9f47c8a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:58:19,595::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:58:19,599::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8cd8a35c-be32-4df4-b58d-dcbc5f89ba8a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:58:19,601::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:58:19,603::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:58:19,605::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8cd8a35c-be32-4df4-b58d-dcbc5f89ba8a`::Granted request >Thread-34::DEBUG::2014-08-11 14:58:19,613::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`be53a31b-f11d-4503-9521-e620f9f47c8a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:58:19,615::task::993::Storage.TaskManager.Task::(_decref) Task=`be53a31b-f11d-4503-9521-e620f9f47c8a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:58:19,667::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858737733632', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:19,669::task::1191::Storage.TaskManager.Task::(prepare) Task=`be53a31b-f11d-4503-9521-e620f9f47c8a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858737733632', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:19,670::task::595::Storage.TaskManager.Task::(_updateState) Task=`be53a31b-f11d-4503-9521-e620f9f47c8a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:58:19,671::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:58:19,672::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:58:19,674::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:58:19,676::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:58:19,677::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:58:19,678::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:58:19,680::task::993::Storage.TaskManager.Task::(_decref) Task=`be53a31b-f11d-4503-9521-e620f9f47c8a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:58:20,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:20,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132644 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:23,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:23,035::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135149 s, 75.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:24,296::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:24,298::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:24,392::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:58:25,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:25,091::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013171 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:26,560::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:26,561::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:26,561::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:26,561::__init__::232::IOProcess::(_processLogs) (6078) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:26,561::__init__::232::IOProcess::(_processLogs) (6078) Queuing response >Thread-16::DEBUG::2014-08-11 14:58:26,562::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:58:26,581::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000438004 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:26,582::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:26,583::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:26,583::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:26,583::__init__::232::IOProcess::(_processLogs) (6079) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:26,584::__init__::232::IOProcess::(_processLogs) DEBUG|(6079) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:26,586::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:26,586::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:26,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:26,587::__init__::232::IOProcess::(_processLogs) (6080) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:26,587::__init__::232::IOProcess::(_processLogs) (6080) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:26,588::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:26,588::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:26,588::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:26,589::__init__::232::IOProcess::(_processLogs) (6081) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:26,589::__init__::232::IOProcess::(_processLogs) DEBUG|(6081) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:26,590::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:26,591::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:26,591::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:26,592::__init__::232::IOProcess::(_processLogs) (6082) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:26,592::__init__::232::IOProcess::(_processLogs) (6082) Queuing response >Dummy-33::DEBUG::2014-08-11 14:58:27,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:27,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134037 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:29,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:29,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131921 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:29,398::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:29,399::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:29,492::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:29,695::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:58:29,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`0284b552-b1f2-4d70-8a99-dcadea8a45f5`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:29,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:58:29,733::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:29,734::task::1191::Storage.TaskManager.Task::(prepare) Task=`0284b552-b1f2-4d70-8a99-dcadea8a45f5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:29,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`0284b552-b1f2-4d70-8a99-dcadea8a45f5`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:29,735::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:29,735::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:29,735::task::993::Storage.TaskManager.Task::(_decref) Task=`0284b552-b1f2-4d70-8a99-dcadea8a45f5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:58:29,752::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:58:29,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`67a0b329-24ca-425d-ba79-9517f8d42a3c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:58:29,756::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:58:29,759::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6eda6e57-0b51-4284-b7b0-340f9a58008d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:58:29,760::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:58:29,762::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:58:29,763::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6eda6e57-0b51-4284-b7b0-340f9a58008d`::Granted request >Thread-34::DEBUG::2014-08-11 14:58:29,766::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`67a0b329-24ca-425d-ba79-9517f8d42a3c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:58:29,768::task::993::Storage.TaskManager.Task::(_decref) Task=`67a0b329-24ca-425d-ba79-9517f8d42a3c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:58:29,815::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858519302144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:29,816::task::1191::Storage.TaskManager.Task::(prepare) Task=`67a0b329-24ca-425d-ba79-9517f8d42a3c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858519302144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:29,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`67a0b329-24ca-425d-ba79-9517f8d42a3c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:58:29,819::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:58:29,820::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:58:29,822::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:58:29,823::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:58:29,824::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:58:29,826::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:58:29,829::task::993::Storage.TaskManager.Task::(_decref) Task=`67a0b329-24ca-425d-ba79-9517f8d42a3c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:58:31,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:31,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147102 s, 69.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:33,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:33,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132309 s, 77.4 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:58:34,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`76aeda6e-8755-4aa2-841f-67996fabd149`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:34,484::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:58:34,491::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000438004', 'lastCheck': '7.9', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:34,492::task::1191::Storage.TaskManager.Task::(prepare) Task=`76aeda6e-8755-4aa2-841f-67996fabd149`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000438004', 'lastCheck': '7.9', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:34,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`76aeda6e-8755-4aa2-841f-67996fabd149`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:34,492::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:34,492::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:34,493::task::993::Storage.TaskManager.Task::(_decref) Task=`76aeda6e-8755-4aa2-841f-67996fabd149`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:34,499::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:34,500::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:34,502::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:34,513::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:58:35,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:35,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133168 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:36,592::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:36,593::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:36,593::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:36,593::__init__::232::IOProcess::(_processLogs) (6083) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:36,593::__init__::232::IOProcess::(_processLogs) (6083) Queuing response >Thread-16::DEBUG::2014-08-11 14:58:36,594::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:58:36,613::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425529 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:36,615::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:36,615::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:36,615::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:36,616::__init__::232::IOProcess::(_processLogs) (6084) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:36,616::__init__::232::IOProcess::(_processLogs) DEBUG|(6084) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:36,618::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:36,618::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:36,618::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:36,619::__init__::232::IOProcess::(_processLogs) DEBUG|(6085) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:36,620::__init__::232::IOProcess::(_processLogs) (6085) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:36,620::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:36,621::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:36,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:36,622::__init__::232::IOProcess::(_processLogs) (6086) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:36,622::__init__::232::IOProcess::(_processLogs) (6086) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:36,622::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:36,623::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:36,623::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:36,624::__init__::232::IOProcess::(_processLogs) (6087) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:36,625::__init__::232::IOProcess::(_processLogs) (6087) Queuing response >Dummy-33::DEBUG::2014-08-11 14:58:37,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:37,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131547 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:39,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:39,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132826 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:39,509::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:39,510::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:39,592::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:39,845::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:58:39,847::task::595::Storage.TaskManager.Task::(_updateState) Task=`592bac70-a9b6-4d4a-89b3-711765167757`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:39,847::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:58:39,869::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:39,872::task::1191::Storage.TaskManager.Task::(prepare) Task=`592bac70-a9b6-4d4a-89b3-711765167757`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:39,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`592bac70-a9b6-4d4a-89b3-711765167757`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:39,872::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:39,873::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:39,873::task::993::Storage.TaskManager.Task::(_decref) Task=`592bac70-a9b6-4d4a-89b3-711765167757`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:58:39,887::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:58:39,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`e56ae7e7-dcfc-4ea5-81e7-9b07ba5f01ab`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:58:39,888::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:58:39,889::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`685debe1-652c-400e-b15e-63e9e0f36349`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:58:39,889::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:58:39,889::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:58:39,890::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`685debe1-652c-400e-b15e-63e9e0f36349`::Granted request >Thread-34::DEBUG::2014-08-11 14:58:39,892::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e56ae7e7-dcfc-4ea5-81e7-9b07ba5f01ab`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:58:39,894::task::993::Storage.TaskManager.Task::(_decref) Task=`e56ae7e7-dcfc-4ea5-81e7-9b07ba5f01ab`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:58:39,949::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858277605376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:39,950::task::1191::Storage.TaskManager.Task::(prepare) Task=`e56ae7e7-dcfc-4ea5-81e7-9b07ba5f01ab`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858277605376', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:39,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`e56ae7e7-dcfc-4ea5-81e7-9b07ba5f01ab`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:58:39,952::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:58:39,952::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:58:39,954::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:58:39,956::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:58:39,957::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:58:39,958::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:58:39,960::task::993::Storage.TaskManager.Task::(_decref) Task=`e56ae7e7-dcfc-4ea5-81e7-9b07ba5f01ab`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:58:41,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:41,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013131 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:43,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:43,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177587 s, 57.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:44,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:44,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:44,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:58:45,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:45,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135922 s, 75.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:46,625::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:46,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:58:46,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:58:46,627::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:46,646::__init__::232::IOProcess::(_processLogs) (6088) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:46,647::__init__::232::IOProcess::(_processLogs) (6088) Queuing response >Thread-16::DEBUG::2014-08-11 14:58:46,647::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000464933 s, 979 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:46,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:46,649::__init__::232::IOProcess::(_processLogs) (6089) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:46,650::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:46,651::__init__::232::IOProcess::(_processLogs) (6090) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:46,651::__init__::232::IOProcess::(_processLogs) DEBUG|(6090) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:46,652::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:46,653::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:46,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:46,653::__init__::232::IOProcess::(_processLogs) (6091) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:46,654::__init__::232::IOProcess::(_processLogs) (6091) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:46,654::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:46,655::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:46,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:46,656::__init__::232::IOProcess::(_processLogs) (6092) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:46,656::__init__::232::IOProcess::(_processLogs) (6092) Queuing response >Dummy-33::DEBUG::2014-08-11 14:58:47,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:47,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013329 s, 76.8 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:58:49,615::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f98d4e4-f198-4ff1-9a19-c7dd18e2bd4d`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:49,615::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:58:49,692::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464933', 'lastCheck': '3.0', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:49,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`7f98d4e4-f198-4ff1-9a19-c7dd18e2bd4d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000464933', 'lastCheck': '3.0', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:58:49,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`7f98d4e4-f198-4ff1-9a19-c7dd18e2bd4d`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:49,692::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:49,692::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:49,693::task::993::Storage.TaskManager.Task::(_decref) Task=`7f98d4e4-f198-4ff1-9a19-c7dd18e2bd4d`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:49,698::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:49,699::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:49,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:58:49,711::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:58:49,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:49,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013218 s, 77.5 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:58:49,982::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:58:49,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`04c7ce9b-1672-41f6-8e7f-b3067b6cfb27`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:58:49,985::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:58:50,016::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:50,017::task::1191::Storage.TaskManager.Task::(prepare) Task=`04c7ce9b-1672-41f6-8e7f-b3067b6cfb27`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:58:50,018::task::595::Storage.TaskManager.Task::(_updateState) Task=`04c7ce9b-1672-41f6-8e7f-b3067b6cfb27`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:58:50,018::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:58:50,018::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:58:50,019::task::993::Storage.TaskManager.Task::(_decref) Task=`04c7ce9b-1672-41f6-8e7f-b3067b6cfb27`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:58:50,036::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:58:50,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a19ca9c-3e57-4cd9-a664-400554115019`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:58:50,041::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:58:50,044::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0da9e8be-3ca1-4b77-80ba-35d44efb9228`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:58:50,045::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:58:50,047::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:58:50,048::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0da9e8be-3ca1-4b77-80ba-35d44efb9228`::Granted request >Thread-34::DEBUG::2014-08-11 14:58:50,051::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5a19ca9c-3e57-4cd9-a664-400554115019`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:58:50,053::task::993::Storage.TaskManager.Task::(_decref) Task=`5a19ca9c-3e57-4cd9-a664-400554115019`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:58:50,119::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858277408768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:50,120::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a19ca9c-3e57-4cd9-a664-400554115019`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858277408768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:58:50,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a19ca9c-3e57-4cd9-a664-400554115019`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:58:50,122::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:58:50,123::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:58:50,125::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:58:50,127::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:58:50,128::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:58:50,129::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:58:50,130::task::993::Storage.TaskManager.Task::(_decref) Task=`5a19ca9c-3e57-4cd9-a664-400554115019`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:58:51,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:51,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133667 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:58:53,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:53,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131469 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:54,706::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:54,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:54,792::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:58:55,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:55,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131386 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:56,657::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:56,657::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:56,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:56,658::__init__::232::IOProcess::(_processLogs) (6093) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:56,658::__init__::232::IOProcess::(_processLogs) (6093) Queuing response >Thread-16::DEBUG::2014-08-11 14:58:56,659::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:58:56,678::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418869 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:58:56,679::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:56,679::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:56,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:56,680::__init__::232::IOProcess::(_processLogs) (6094) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:58:56,680::__init__::232::IOProcess::(_processLogs) DEBUG|(6094) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:56,682::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:56,682::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:56,683::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:56,683::__init__::232::IOProcess::(_processLogs) (6095) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:56,683::__init__::232::IOProcess::(_processLogs) (6095) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:56,684::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:56,684::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:56,685::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:56,685::__init__::232::IOProcess::(_processLogs) (6096) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:56,686::__init__::232::IOProcess::(_processLogs) (6096) Queuing response >Thread-14::DEBUG::2014-08-11 14:58:56,686::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:58:56,686::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:58:56,687::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:58:56,688::__init__::232::IOProcess::(_processLogs) (6097) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:58:56,688::__init__::232::IOProcess::(_processLogs) (6097) Queuing response >Dummy-33::DEBUG::2014-08-11 14:58:57,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:58:57,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134782 s, 76.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:59,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:59,797::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:58:59,892::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:59:00,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:00,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133607 s, 76.6 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:59:00,154::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-106::DEBUG::2014-08-11 14:59:00,157::task::595::Storage.TaskManager.Task::(_updateState) Task=`c58c4784-62b6-4418-906b-3f23e9112cda`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:00,157::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:59:00,176::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:00,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`c58c4784-62b6-4418-906b-3f23e9112cda`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:00,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`c58c4784-62b6-4418-906b-3f23e9112cda`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:00,176::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:00,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:00,177::task::993::Storage.TaskManager.Task::(_decref) Task=`c58c4784-62b6-4418-906b-3f23e9112cda`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:59:00,190::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-34::DEBUG::2014-08-11 14:59:00,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`184c7936-8b52-49fe-b9af-f685792b5984`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:59:00,194::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:59:00,198::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3ccac6a9-62af-4453-bb08-8701021b0d5a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:59:00,199::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:59:00,200::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:59:00,202::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3ccac6a9-62af-4453-bb08-8701021b0d5a`::Granted request >Thread-34::DEBUG::2014-08-11 14:59:00,205::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`184c7936-8b52-49fe-b9af-f685792b5984`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:59:00,206::task::993::Storage.TaskManager.Task::(_decref) Task=`184c7936-8b52-49fe-b9af-f685792b5984`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:59:00,252::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:00,253::task::1191::Storage.TaskManager.Task::(prepare) Task=`184c7936-8b52-49fe-b9af-f685792b5984`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:00,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`184c7936-8b52-49fe-b9af-f685792b5984`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:59:00,255::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:59:00,256::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:59:00,258::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:59:00,260::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:59:00,261::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:59:00,262::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:59:00,265::task::993::Storage.TaskManager.Task::(_decref) Task=`184c7936-8b52-49fe-b9af-f685792b5984`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:59:02,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:02,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277716 s, 36.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:04,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:04,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131804 s, 77.7 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:59:04,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0626ce6-ccb6-44f8-9702-f9fa9b76a687`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:04,800::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:59:04,892::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418869', 'lastCheck': '8.2', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:04,892::task::1191::Storage.TaskManager.Task::(prepare) Task=`f0626ce6-ccb6-44f8-9702-f9fa9b76a687`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418869', 'lastCheck': '8.2', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:04,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0626ce6-ccb6-44f8-9702-f9fa9b76a687`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:04,892::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:04,893::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:04,893::task::993::Storage.TaskManager.Task::(_decref) Task=`f0626ce6-ccb6-44f8-9702-f9fa9b76a687`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:04,898::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:04,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:04,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:04,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:59:06,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:06,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132282 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:06,689::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:06,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:59:06,691::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:59:06,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:06,709::__init__::232::IOProcess::(_processLogs) (6098) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:06,709::__init__::232::IOProcess::(_processLogs) (6098) Queuing response >Thread-16::DEBUG::2014-08-11 14:59:06,710::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000405924 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:06,711::__init__::232::IOProcess::(_processLogs) (6099) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:06,713::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:06,713::__init__::232::IOProcess::(_processLogs) (6100) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:06,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:06,714::__init__::232::IOProcess::(_processLogs) (6101) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:06,715::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6101) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:06,716::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:06,716::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:06,717::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:06,718::__init__::232::IOProcess::(_processLogs) (6102) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:06,718::__init__::232::IOProcess::(_processLogs) (6102) Queuing response >Dummy-33::DEBUG::2014-08-11 14:59:08,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:08,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131946 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:09,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:09,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:09,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:10,282::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:59:10,283::task::595::Storage.TaskManager.Task::(_updateState) Task=`5fb0a592-35a3-4a11-9d45-7df9872aea15`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:10,283::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:59:10,311::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:10,312::task::1191::Storage.TaskManager.Task::(prepare) Task=`5fb0a592-35a3-4a11-9d45-7df9872aea15`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:10,312::task::595::Storage.TaskManager.Task::(_updateState) Task=`5fb0a592-35a3-4a11-9d45-7df9872aea15`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:10,312::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:10,312::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:10,312::task::993::Storage.TaskManager.Task::(_decref) Task=`5fb0a592-35a3-4a11-9d45-7df9872aea15`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:59:10,323::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:59:10,324::task::595::Storage.TaskManager.Task::(_updateState) Task=`639eefff-c684-44ce-954d-b7a4a745d86e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:59:10,325::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:59:10,325::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b627f607-73af-4129-951e-b975fb173e2f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:59:10,326::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:59:10,326::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:59:10,326::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b627f607-73af-4129-951e-b975fb173e2f`::Granted request >Thread-34::DEBUG::2014-08-11 14:59:10,327::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`639eefff-c684-44ce-954d-b7a4a745d86e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:59:10,327::task::993::Storage.TaskManager.Task::(_decref) Task=`639eefff-c684-44ce-954d-b7a4a745d86e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:59:10,357::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:10,358::task::1191::Storage.TaskManager.Task::(prepare) Task=`639eefff-c684-44ce-954d-b7a4a745d86e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:10,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`639eefff-c684-44ce-954d-b7a4a745d86e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:59:10,360::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:59:10,361::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:59:10,364::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:59:10,366::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:59:10,367::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:59:10,369::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:59:10,372::task::993::Storage.TaskManager.Task::(_decref) Task=`639eefff-c684-44ce-954d-b7a4a745d86e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:59:10,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:10,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131999 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:12,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:12,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130689 s, 78.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 14:59:14,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a9f767e-14b9-40ac-b6fb-185ff7e74862`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 14:59:14,529::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 14:59:14,530::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:14,530::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:14,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:14,531::__init__::232::IOProcess::(_processLogs) (6103) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:59:14,531::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6103) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:14,532::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:14,533::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:14,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:14,533::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6104) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 14:59:14,533::__init__::232::IOProcess::(_processLogs) (6104) Queuing response >Thread-55::INFO::2014-08-11 14:59:14,534::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:59:14,534::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a9f767e-14b9-40ac-b6fb-185ff7e74862`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 14:59:14,534::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a9f767e-14b9-40ac-b6fb-185ff7e74862`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 14:59:14,534::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 14:59:14,535::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 14:59:14,535::task::993::Storage.TaskManager.Task::(_decref) Task=`2a9f767e-14b9-40ac-b6fb-185ff7e74862`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:59:14,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:14,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015053 s, 68.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:14,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:15,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:15,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:59:16,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:16,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132105 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:16,719::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:16,719::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:16,719::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:16,719::__init__::232::IOProcess::(_processLogs) (6105) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:16,720::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6105) Queuing response >Thread-16::DEBUG::2014-08-11 14:59:16,721::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:59:16,739::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420091 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:16,741::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:16,741::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:16,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:16,741::__init__::232::IOProcess::(_processLogs) (6106) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:16,742::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6106) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:16,744::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:16,744::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:16,744::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:16,744::__init__::232::IOProcess::(_processLogs) (6107) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:16,745::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6107) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:16,746::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:16,746::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:16,747::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:16,747::__init__::232::IOProcess::(_processLogs) (6108) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:16,748::__init__::232::IOProcess::(_processLogs) (6108) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:16,748::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:16,748::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:16,748::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:16,749::__init__::232::IOProcess::(_processLogs) (6109) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:16,749::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6109) Queuing response >Dummy-33::DEBUG::2014-08-11 14:59:18,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:18,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131581 s, 77.8 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:59:20,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`59fe6df5-2d10-4a68-aa08-ee3a6f94854b`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:20,004::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:59:20,092::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420091', 'lastCheck': '3.3', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:20,092::task::1191::Storage.TaskManager.Task::(prepare) Task=`59fe6df5-2d10-4a68-aa08-ee3a6f94854b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420091', 'lastCheck': '3.3', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:20,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`59fe6df5-2d10-4a68-aa08-ee3a6f94854b`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:20,092::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:20,093::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:20,093::task::993::Storage.TaskManager.Task::(_decref) Task=`59fe6df5-2d10-4a68-aa08-ee3a6f94854b`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:20,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:20,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:20,101::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:20,112::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:20,391::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:59:20,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bae69b5-52b6-4cb3-92b1-5b638a292871`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:20,392::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:59:20,411::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:20,413::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bae69b5-52b6-4cb3-92b1-5b638a292871`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:20,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bae69b5-52b6-4cb3-92b1-5b638a292871`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:20,414::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:20,414::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:20,414::task::993::Storage.TaskManager.Task::(_decref) Task=`8bae69b5-52b6-4cb3-92b1-5b638a292871`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:59:20,444::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:59:20,452::task::595::Storage.TaskManager.Task::(_updateState) Task=`efeac7c3-252f-45a2-82d7-3da34acf12e2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:59:20,453::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:59:20,455::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9177ecce-7933-4475-9067-6f355c3cde76`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:59:20,455::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:59:20,459::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:59:20,462::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9177ecce-7933-4475-9067-6f355c3cde76`::Granted request >Thread-34::DEBUG::2014-08-11 14:59:20,466::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`efeac7c3-252f-45a2-82d7-3da34acf12e2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:59:20,467::task::993::Storage.TaskManager.Task::(_decref) Task=`efeac7c3-252f-45a2-82d7-3da34acf12e2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:59:20,529::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:20,530::task::1191::Storage.TaskManager.Task::(prepare) Task=`efeac7c3-252f-45a2-82d7-3da34acf12e2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:20,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`efeac7c3-252f-45a2-82d7-3da34acf12e2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:59:20,534::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:59:20,535::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:59:20,538::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:59:20,540::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:59:20,542::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:59:20,543::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:59:20,546::task::993::Storage.TaskManager.Task::(_decref) Task=`efeac7c3-252f-45a2-82d7-3da34acf12e2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:59:20,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:20,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133068 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:22,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:22,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134479 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:24,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:24,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131611 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:25,108::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:25,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:25,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:59:26,751::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:26,752::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 14:59:26,753::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 14:59:26,771::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:26,772::__init__::232::IOProcess::(_processLogs) (6110) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:26,772::__init__::232::IOProcess::(_processLogs) (6110) Queuing response >Thread-16::DEBUG::2014-08-11 14:59:26,773::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000398608 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:26,774::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 14:59:26,774::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:26,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:26,775::__init__::232::IOProcess::(_processLogs) (6111) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:26,776::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:26,776::__init__::232::IOProcess::(_processLogs) (6112) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:26,777::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:26,777::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:26,778::__init__::232::IOProcess::(_processLogs) (6113) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:26,779::__init__::232::IOProcess::(_processLogs) (6113) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:26,779::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:26,779::__init__::232::IOProcess::(_processLogs) (6114) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:26,780::__init__::232::IOProcess::(_processLogs) (6114) Queuing response >Dummy-33::DEBUG::2014-08-11 14:59:26,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:26,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133062 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:28,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:29,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133403 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:30,197::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:30,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:30,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:30,558::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:59:30,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`7da51314-f3d5-4ad3-be30-e6275dc1c304`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:30,560::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:59:30,572::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:30,573::task::1191::Storage.TaskManager.Task::(prepare) Task=`7da51314-f3d5-4ad3-be30-e6275dc1c304`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:30,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`7da51314-f3d5-4ad3-be30-e6275dc1c304`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:30,573::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:30,573::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:30,574::task::993::Storage.TaskManager.Task::(_decref) Task=`7da51314-f3d5-4ad3-be30-e6275dc1c304`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:59:30,584::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:59:30,584::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b249945-bb80-4bee-8140-6ad26a98075c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:59:30,584::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:59:30,585::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`565e1fe4-6717-44f5-85fc-9ff4765c2892`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:59:30,585::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:59:30,586::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:59:30,586::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`565e1fe4-6717-44f5-85fc-9ff4765c2892`::Granted request >Thread-34::DEBUG::2014-08-11 14:59:30,587::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2b249945-bb80-4bee-8140-6ad26a98075c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:59:30,587::task::993::Storage.TaskManager.Task::(_decref) Task=`2b249945-bb80-4bee-8140-6ad26a98075c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:59:30,614::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:30,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b249945-bb80-4bee-8140-6ad26a98075c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858249752576', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:30,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b249945-bb80-4bee-8140-6ad26a98075c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:59:30,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:59:30,623::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:59:30,624::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:59:30,627::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:59:30,628::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:59:30,628::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:59:30,629::task::993::Storage.TaskManager.Task::(_decref) Task=`2b249945-bb80-4bee-8140-6ad26a98075c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:59:31,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:31,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132523 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:33,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:33,114::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132339 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:35,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:35,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132016 s, 77.6 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:59:35,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`15449099-320c-4f9a-a5c1-356b36915768`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:35,207::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:59:35,292::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398608', 'lastCheck': '8.5', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:35,292::task::1191::Storage.TaskManager.Task::(prepare) Task=`15449099-320c-4f9a-a5c1-356b36915768`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000398608', 'lastCheck': '8.5', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:35,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`15449099-320c-4f9a-a5c1-356b36915768`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:35,293::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:35,293::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:35,293::task::993::Storage.TaskManager.Task::(_decref) Task=`15449099-320c-4f9a-a5c1-356b36915768`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:35,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:35,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:35,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:35,312::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 14:59:36,782::__init__::232::IOProcess::(_processLogs) (6115) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:59:36,783::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:59:36,802::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000406936 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:36,803::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:36,804::__init__::232::IOProcess::(_processLogs) (6116) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:36,805::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:36,805::__init__::232::IOProcess::(_processLogs) (6117) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:36,806::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:36,806::__init__::232::IOProcess::(_processLogs) (6118) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:36,807::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:36,807::__init__::232::IOProcess::(_processLogs) (6119) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:36,808::__init__::232::IOProcess::(_processLogs) (6119) Queuing response >Dummy-33::DEBUG::2014-08-11 14:59:37,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:37,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132134 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:39,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:39,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131432 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:40,309::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:40,310::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:40,311::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:40,642::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:59:40,643::task::595::Storage.TaskManager.Task::(_updateState) Task=`3409bffd-8a37-4945-857a-b155c90c995f`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:40,644::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:59:40,656::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:40,656::task::1191::Storage.TaskManager.Task::(prepare) Task=`3409bffd-8a37-4945-857a-b155c90c995f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:40,657::task::595::Storage.TaskManager.Task::(_updateState) Task=`3409bffd-8a37-4945-857a-b155c90c995f`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:40,657::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:40,657::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:40,657::task::993::Storage.TaskManager.Task::(_decref) Task=`3409bffd-8a37-4945-857a-b155c90c995f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:59:40,667::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:59:40,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`37e22be0-0333-4ace-a6f0-3bc124ec18eb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:59:40,668::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:59:40,669::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cc864834-1aaa-459c-8fc2-40a9d678fe62`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:59:40,669::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:59:40,669::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:59:40,670::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cc864834-1aaa-459c-8fc2-40a9d678fe62`::Granted request >Thread-34::DEBUG::2014-08-11 14:59:40,670::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`37e22be0-0333-4ace-a6f0-3bc124ec18eb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:59:40,670::task::993::Storage.TaskManager.Task::(_decref) Task=`37e22be0-0333-4ace-a6f0-3bc124ec18eb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:59:40,690::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:40,691::task::1191::Storage.TaskManager.Task::(prepare) Task=`37e22be0-0333-4ace-a6f0-3bc124ec18eb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:40,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`37e22be0-0333-4ace-a6f0-3bc124ec18eb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:59:40,691::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:59:40,691::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:59:40,692::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:59:40,692::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:59:40,692::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:59:40,692::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:59:40,693::task::993::Storage.TaskManager.Task::(_decref) Task=`37e22be0-0333-4ace-a6f0-3bc124ec18eb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:59:41,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:41,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132706 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:43,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:43,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132869 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:45,318::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:45,319::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:59:45,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:45,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:59:45,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133202 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:46,812::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:46,812::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:46,812::__init__::232::IOProcess::(_processLogs) (6120) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:59:46,813::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:59:46,832::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432766 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:46,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:46,834::__init__::232::IOProcess::(_processLogs) (6121) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:46,834::__init__::232::IOProcess::(_processLogs) (6121) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:46,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:46,836::__init__::232::IOProcess::(_processLogs) (6122) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:46,836::__init__::232::IOProcess::(_processLogs) (6122) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:46,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:46,837::__init__::232::IOProcess::(_processLogs) (6123) Queuing response >Thread-14::DEBUG::2014-08-11 14:59:46,838::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:46,838::__init__::232::IOProcess::(_processLogs) (6124) Queuing response >Dummy-33::DEBUG::2014-08-11 14:59:47,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:47,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132747 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:49,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:49,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013208 s, 77.5 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 14:59:50,410::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e92197c-37a6-43c2-937c-01739f3b7980`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:50,410::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 14:59:50,492::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432766', 'lastCheck': '3.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:50,492::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e92197c-37a6-43c2-937c-01739f3b7980`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432766', 'lastCheck': '3.7', 'valid': True}} >Thread-106::DEBUG::2014-08-11 14:59:50,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e92197c-37a6-43c2-937c-01739f3b7980`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:50,492::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:50,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:50,493::task::993::Storage.TaskManager.Task::(_decref) Task=`7e92197c-37a6-43c2-937c-01739f3b7980`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:50,518::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:50,521::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:50,524::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:50,529::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 14:59:50,706::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 14:59:50,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e76acc2-a827-4c5a-b8e8-bdbce795154f`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 14:59:50,707::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 14:59:50,719::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:50,720::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e76acc2-a827-4c5a-b8e8-bdbce795154f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 14:59:50,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e76acc2-a827-4c5a-b8e8-bdbce795154f`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 14:59:50,720::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 14:59:50,720::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 14:59:50,720::task::993::Storage.TaskManager.Task::(_decref) Task=`5e76acc2-a827-4c5a-b8e8-bdbce795154f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 14:59:50,731::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 14:59:50,732::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e8666d5-5083-4e81-a1bd-d1d079185fc1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 14:59:50,732::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 14:59:50,733::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3958954a-9695-4b2a-9966-3f8a8ad48c87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 14:59:50,733::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 14:59:50,734::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 14:59:50,734::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3958954a-9695-4b2a-9966-3f8a8ad48c87`::Granted request >Thread-34::DEBUG::2014-08-11 14:59:50,734::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2e8666d5-5083-4e81-a1bd-d1d079185fc1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 14:59:50,735::task::993::Storage.TaskManager.Task::(_decref) Task=`2e8666d5-5083-4e81-a1bd-d1d079185fc1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 14:59:50,754::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:50,755::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e8666d5-5083-4e81-a1bd-d1d079185fc1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 14:59:50,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e8666d5-5083-4e81-a1bd-d1d079185fc1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 14:59:50,755::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 14:59:50,755::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 14:59:50,756::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 14:59:50,756::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 14:59:50,756::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 14:59:50,756::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 14:59:50,757::task::993::Storage.TaskManager.Task::(_decref) Task=`2e8666d5-5083-4e81-a1bd-d1d079185fc1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 14:59:51,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:51,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132703 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:53,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:53,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131716 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:55,538::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:55,540::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 14:59:55,592::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 14:59:55,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:55,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132239 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:56,840::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:56,841::__init__::232::IOProcess::(_processLogs) (6125) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 14:59:56,842::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 14:59:56,868::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000560354 s, 812 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 14:59:56,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 14:59:56,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 14:59:56,872::__init__::232::IOProcess::(_processLogs) (6126) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 14:59:56,874::__init__::232::IOProcess::(_processLogs) (6127) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:56,875::__init__::232::IOProcess::(_processLogs) (6128) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 14:59:56,876::__init__::232::IOProcess::(_processLogs) (6129) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 14:59:57,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:57,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132087 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 14:59:59,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 14:59:59,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134264 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:00,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:00,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:00,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 15:00:00,770::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 15:00:00,771::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b78c845-5975-4245-b996-9098063dd96b`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 15:00:00,771::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 15:00:00,784::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 15:00:00,785::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b78c845-5975-4245-b996-9098063dd96b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 15:00:00,785::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b78c845-5975-4245-b996-9098063dd96b`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 15:00:00,785::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 15:00:00,785::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 15:00:00,785::task::993::Storage.TaskManager.Task::(_decref) Task=`9b78c845-5975-4245-b996-9098063dd96b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:00:00,801::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:00:00,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`360919d7-4e77-4633-8d2a-a9581420c17c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:00:00,802::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:00:00,805::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`256eef76-f13a-4ec9-b871-4013abc5cd39`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:00:00,805::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:00:00,805::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:00:00,806::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`256eef76-f13a-4ec9-b871-4013abc5cd39`::Granted request >Thread-34::DEBUG::2014-08-11 15:00:00,806::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`360919d7-4e77-4633-8d2a-a9581420c17c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:00:00,807::task::993::Storage.TaskManager.Task::(_decref) Task=`360919d7-4e77-4633-8d2a-a9581420c17c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:00:00,826::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:00,827::task::1191::Storage.TaskManager.Task::(prepare) Task=`360919d7-4e77-4633-8d2a-a9581420c17c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:00,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`360919d7-4e77-4633-8d2a-a9581420c17c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:00:00,827::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:00:00,828::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:00:00,828::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:00:00,828::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:00:00,828::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:00:00,829::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:00:00,829::task::993::Storage.TaskManager.Task::(_decref) Task=`360919d7-4e77-4633-8d2a-a9581420c17c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:00:01,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:01,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265632 s, 38.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:04,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:04,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132616 s, 77.2 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 15:00:05,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdbcd0d7-9e4e-4c60-bde3-f77319a822b6`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 15:00:05,625::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 15:00:05,692::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000560354', 'lastCheck': '8.8', 'valid': True}} >Thread-106::DEBUG::2014-08-11 15:00:05,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdbcd0d7-9e4e-4c60-bde3-f77319a822b6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000560354', 'lastCheck': '8.8', 'valid': True}} >Thread-106::DEBUG::2014-08-11 15:00:05,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdbcd0d7-9e4e-4c60-bde3-f77319a822b6`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 15:00:05,692::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 15:00:05,693::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 15:00:05,693::task::993::Storage.TaskManager.Task::(_decref) Task=`bdbcd0d7-9e4e-4c60-bde3-f77319a822b6`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:05,699::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:05,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:05,701::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 15:00:05,712::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:00:06,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:06,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131676 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:06,878::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:06,878::__init__::232::IOProcess::(_processLogs) (6130) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:00:06,879::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:00:06,898::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418693 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:06,899::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:06,899::__init__::232::IOProcess::(_processLogs) (6131) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:06,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:06,901::__init__::232::IOProcess::(_processLogs) (6132) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:06,901::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6132) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:06,902::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:06,902::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:06,903::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:06,903::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6133) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:06,903::__init__::232::IOProcess::(_processLogs) (6133) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:06,904::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:06,904::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:06,904::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:06,905::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6134) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:06,905::__init__::232::IOProcess::(_processLogs) (6134) Queuing response >Dummy-33::DEBUG::2014-08-11 15:00:08,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:08,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132686 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:10,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:10,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132525 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:10,709::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:10,710::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:10,792::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 15:00:10,842::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 15:00:10,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`463d83a6-f231-4488-8afa-0174521793bd`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 15:00:10,843::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 15:00:10,856::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 15:00:10,857::task::1191::Storage.TaskManager.Task::(prepare) Task=`463d83a6-f231-4488-8afa-0174521793bd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 15:00:10,857::task::595::Storage.TaskManager.Task::(_updateState) Task=`463d83a6-f231-4488-8afa-0174521793bd`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 15:00:10,857::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 15:00:10,857::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 15:00:10,857::task::993::Storage.TaskManager.Task::(_decref) Task=`463d83a6-f231-4488-8afa-0174521793bd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:00:10,869::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:00:10,870::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc1ff0ca-560a-41a9-91ef-7d0db865e6b6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:00:10,870::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:00:10,871::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aadd2fa8-7da8-4772-97fc-32e95cf20a9f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:00:10,871::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:00:10,872::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:00:10,872::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aadd2fa8-7da8-4772-97fc-32e95cf20a9f`::Granted request >Thread-34::DEBUG::2014-08-11 15:00:10,872::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bc1ff0ca-560a-41a9-91ef-7d0db865e6b6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:00:10,873::task::993::Storage.TaskManager.Task::(_decref) Task=`bc1ff0ca-560a-41a9-91ef-7d0db865e6b6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:00:10,893::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:10,893::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc1ff0ca-560a-41a9-91ef-7d0db865e6b6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:10,894::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc1ff0ca-560a-41a9-91ef-7d0db865e6b6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:00:10,894::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:00:10,894::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:00:10,894::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:00:10,895::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:00:10,895::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:00:10,895::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:00:10,895::task::993::Storage.TaskManager.Task::(_decref) Task=`bc1ff0ca-560a-41a9-91ef-7d0db865e6b6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:00:12,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:12,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132798 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:14,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:14,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133337 s, 76.8 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:00:14,669::task::595::Storage.TaskManager.Task::(_updateState) Task=`54d6a784-eb2c-4574-858b-c437323fc559`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:00:14,670::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:00:14,671::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:14,671::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:14,671::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:14,672::__init__::232::IOProcess::(_processLogs) (6135) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:00:14,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:14,673::__init__::232::IOProcess::(_processLogs) (6136) Queuing response >Thread-55::INFO::2014-08-11 15:00:14,674::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:00:14,674::task::1191::Storage.TaskManager.Task::(prepare) Task=`54d6a784-eb2c-4574-858b-c437323fc559`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:00:14,674::task::595::Storage.TaskManager.Task::(_updateState) Task=`54d6a784-eb2c-4574-858b-c437323fc559`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:00:14,675::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:00:14,675::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:00:14,675::task::993::Storage.TaskManager.Task::(_decref) Task=`54d6a784-eb2c-4574-858b-c437323fc559`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:15,798::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:15,892::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:15,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:00:16,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:16,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134804 s, 76.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:16,907::__init__::232::IOProcess::(_processLogs) (6137) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:00:16,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:00:16,927::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422634 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:16,929::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:16,929::__init__::232::IOProcess::(_processLogs) (6138) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:16,930::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:16,931::__init__::232::IOProcess::(_processLogs) (6139) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:16,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:16,932::__init__::232::IOProcess::(_processLogs) (6140) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:16,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:16,933::__init__::232::IOProcess::(_processLogs) (6141) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:00:18,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:18,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131512 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:20,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:20,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132418 s, 77.3 MB/s\n'; <rc> = 0 >Thread-106::DEBUG::2014-08-11 15:00:20,805::task::595::Storage.TaskManager.Task::(_updateState) Task=`18916e09-730c-4bad-8d01-a1e79fd999b6`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 15:00:20,805::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-106::INFO::2014-08-11 15:00:20,892::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422634', 'lastCheck': '4.0', 'valid': True}} >Thread-106::DEBUG::2014-08-11 15:00:20,892::task::1191::Storage.TaskManager.Task::(prepare) Task=`18916e09-730c-4bad-8d01-a1e79fd999b6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422634', 'lastCheck': '4.0', 'valid': True}} >Thread-106::DEBUG::2014-08-11 15:00:20,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`18916e09-730c-4bad-8d01-a1e79fd999b6`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 15:00:20,892::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 15:00:20,893::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 15:00:20,893::task::993::Storage.TaskManager.Task::(_decref) Task=`18916e09-730c-4bad-8d01-a1e79fd999b6`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:20,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:20,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:20,902::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-106::DEBUG::2014-08-11 15:00:20,910::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-106::DEBUG::2014-08-11 15:00:20,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c155304-ce77-40d3-afe5-52fc766a38ea`::moving from state init -> state preparing >Thread-106::INFO::2014-08-11 15:00:20,911::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-106::INFO::2014-08-11 15:00:20,924::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 15:00:20,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c155304-ce77-40d3-afe5-52fc766a38ea`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-106::DEBUG::2014-08-11 15:00:20,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c155304-ce77-40d3-afe5-52fc766a38ea`::moving from state preparing -> state finished >Thread-106::DEBUG::2014-08-11 15:00:20,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-106::DEBUG::2014-08-11 15:00:20,925::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-106::DEBUG::2014-08-11 15:00:20,925::task::993::Storage.TaskManager.Task::(_decref) Task=`0c155304-ce77-40d3-afe5-52fc766a38ea`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 15:00:20,938::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:45882 >Detector thread::DEBUG::2014-08-11 15:00:20,938::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:45882 >Detector thread::DEBUG::2014-08-11 15:00:20,939::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:45882 >Detector thread::DEBUG::2014-08-11 15:00:20,939::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 45882) >Thread-108::DEBUG::2014-08-11 15:00:20,942::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:00:20,950::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:00:20,951::task::595::Storage.TaskManager.Task::(_updateState) Task=`c68accba-e49c-4019-b4bf-c306a76cf584`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:00:20,951::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:00:20,952::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`69f2ec8b-a86a-4950-a109-eb3485b15950`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:00:20,952::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:00:20,952::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:00:20,952::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`69f2ec8b-a86a-4950-a109-eb3485b15950`::Granted request >Thread-34::DEBUG::2014-08-11 15:00:20,953::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c68accba-e49c-4019-b4bf-c306a76cf584`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:00:20,953::task::993::Storage.TaskManager.Task::(_decref) Task=`c68accba-e49c-4019-b4bf-c306a76cf584`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:00:20,973::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:20,974::task::1191::Storage.TaskManager.Task::(prepare) Task=`c68accba-e49c-4019-b4bf-c306a76cf584`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:20,974::task::595::Storage.TaskManager.Task::(_updateState) Task=`c68accba-e49c-4019-b4bf-c306a76cf584`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:00:20,974::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:00:20,974::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:00:20,975::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:00:20,975::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:00:20,975::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:00:20,975::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:00:20,976::task::993::Storage.TaskManager.Task::(_decref) Task=`c68accba-e49c-4019-b4bf-c306a76cf584`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:00:22,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:22,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01321 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:24,717::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:24,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131547 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:25,910::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:25,911::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:25,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:00:26,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:26,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131648 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:26,935::__init__::232::IOProcess::(_processLogs) (6142) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:00:26,936::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:00:26,955::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000375995 s, 1.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:26,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:26,956::__init__::232::IOProcess::(_processLogs) (6143) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:26,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:26,959::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:26,959::__init__::232::IOProcess::(_processLogs) (6144) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:26,959::__init__::232::IOProcess::(_processLogs) (6144) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:26,960::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:26,961::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:26,961::__init__::232::IOProcess::(_processLogs) (6146) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:26,961::__init__::232::IOProcess::(_processLogs) (6146) Queuing response >Dummy-33::DEBUG::2014-08-11 15:00:28,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:28,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134335 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:30,883::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:30,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132939 s, 77.0 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:00:30,992::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:00:30,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`4208d806-1ad6-4016-bec0-a0d7c9613eef`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:00:30,993::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:30,997::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:30,998::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:30,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::INFO::2014-08-11 15:00:31,006::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:00:31,006::task::1191::Storage.TaskManager.Task::(prepare) Task=`4208d806-1ad6-4016-bec0-a0d7c9613eef`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:00:31,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`4208d806-1ad6-4016-bec0-a0d7c9613eef`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:00:31,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:00:31,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:00:31,007::task::993::Storage.TaskManager.Task::(_decref) Task=`4208d806-1ad6-4016-bec0-a0d7c9613eef`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:00:31,021::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:00:31,022::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e6b74b1-65a2-4291-a8d3-2debdb19f75c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:00:31,024::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:00:31,028::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`480e27ee-bc5c-4361-8cba-92ba0e7b1f02`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:00:31,029::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:00:31,029::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:00:31,030::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`480e27ee-bc5c-4361-8cba-92ba0e7b1f02`::Granted request >Thread-34::DEBUG::2014-08-11 15:00:31,033::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1e6b74b1-65a2-4291-a8d3-2debdb19f75c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:00:31,034::task::993::Storage.TaskManager.Task::(_decref) Task=`1e6b74b1-65a2-4291-a8d3-2debdb19f75c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:00:31,064::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:31,064::task::1191::Storage.TaskManager.Task::(prepare) Task=`1e6b74b1-65a2-4291-a8d3-2debdb19f75c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:31,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e6b74b1-65a2-4291-a8d3-2debdb19f75c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:00:31,065::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:00:31,065::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:00:31,065::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:00:31,066::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:00:31,066::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:00:31,066::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:00:31,066::task::993::Storage.TaskManager.Task::(_decref) Task=`1e6b74b1-65a2-4291-a8d3-2debdb19f75c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:00:32,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:32,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134243 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:34,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:35,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132513 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:36,002::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:36,003::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:00:36,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`71cb7978-3503-48cc-8cd0-0a4d3b6e4137`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:00:36,093::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:00:36,095::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000375995', 'lastCheck': '9.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:36,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:00:36,097::task::1191::Storage.TaskManager.Task::(prepare) Task=`71cb7978-3503-48cc-8cd0-0a4d3b6e4137`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000375995', 'lastCheck': '9.1', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:00:36,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`71cb7978-3503-48cc-8cd0-0a4d3b6e4137`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:00:36,099::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:00:36,100::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:00:36,100::task::993::Storage.TaskManager.Task::(_decref) Task=`71cb7978-3503-48cc-8cd0-0a4d3b6e4137`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:00:36,120::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:00:36,963::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:36,964::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:36,964::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:36,964::__init__::232::IOProcess::(_processLogs) (6147) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:36,965::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6147) Queuing response >Thread-16::DEBUG::2014-08-11 15:00:36,966::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:00:36,984::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000438016 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:36,986::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:36,986::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:36,986::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:36,987::__init__::232::IOProcess::(_processLogs) (6148) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:36,987::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6148) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:36,989::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:36,989::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:36,989::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:36,989::__init__::232::IOProcess::(_processLogs) (6149) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:36,990::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6149) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:36,991::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:36,991::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:36,991::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:36,992::__init__::232::IOProcess::(_processLogs) (6150) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:36,992::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6150) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:36,993::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:36,993::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:36,994::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:36,994::__init__::232::IOProcess::(_processLogs) (6151) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:36,994::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6151) Queuing response >Dummy-33::DEBUG::2014-08-11 15:00:37,049::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:37,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019616 s, 52.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:39,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:39,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132139 s, 77.5 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:00:41,079::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:00:41,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`3322420d-98b2-46d1-966f-47f47a10e24c`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:00:41,091::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:41,106::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:41,107::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:41,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::INFO::2014-08-11 15:00:41,110::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:00:41,110::task::1191::Storage.TaskManager.Task::(prepare) Task=`3322420d-98b2-46d1-966f-47f47a10e24c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:00:41,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`3322420d-98b2-46d1-966f-47f47a10e24c`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:00:41,111::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:00:41,111::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:00:41,111::task::993::Storage.TaskManager.Task::(_decref) Task=`3322420d-98b2-46d1-966f-47f47a10e24c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:00:41,122::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:00:41,122::task::595::Storage.TaskManager.Task::(_updateState) Task=`832bd26f-8736-49fc-af1b-ac452b9ce0f4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:00:41,123::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:00:41,124::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`74e63b31-e14a-4943-8844-5d76b23e2f03`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:00:41,124::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:00:41,124::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:00:41,124::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`74e63b31-e14a-4943-8844-5d76b23e2f03`::Granted request >Thread-34::DEBUG::2014-08-11 15:00:41,125::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`832bd26f-8736-49fc-af1b-ac452b9ce0f4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:00:41,125::task::993::Storage.TaskManager.Task::(_decref) Task=`832bd26f-8736-49fc-af1b-ac452b9ce0f4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:00:41,145::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:41,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`832bd26f-8736-49fc-af1b-ac452b9ce0f4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:41,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`832bd26f-8736-49fc-af1b-ac452b9ce0f4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:00:41,146::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:00:41,146::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:00:41,146::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:00:41,147::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:00:41,147::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:00:41,147::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:00:41,147::task::993::Storage.TaskManager.Task::(_decref) Task=`832bd26f-8736-49fc-af1b-ac452b9ce0f4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:00:41,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:41,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132589 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:43,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:43,258::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133272 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:45,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:45,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132432 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:46,116::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:46,117::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:46,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:00:46,996::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:46,997::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:46,997::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:46,997::__init__::232::IOProcess::(_processLogs) (6152) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:46,998::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6152) Queuing response >Thread-16::DEBUG::2014-08-11 15:00:46,999::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:00:47,018::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419135 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:47,019::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:47,020::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:47,020::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:47,020::__init__::232::IOProcess::(_processLogs) (6153) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:47,021::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6153) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:47,022::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:47,023::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:47,024::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:47,024::__init__::232::IOProcess::(_processLogs) (6154) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:47,024::__init__::232::IOProcess::(_processLogs) (6154) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:47,024::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:47,025::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:47,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:47,026::__init__::232::IOProcess::(_processLogs) (6155) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:47,026::__init__::232::IOProcess::(_processLogs) (6155) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:47,027::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:47,027::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:47,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:47,029::__init__::232::IOProcess::(_processLogs) (6156) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:47,029::__init__::232::IOProcess::(_processLogs) (6156) Queuing response >Dummy-33::DEBUG::2014-08-11 15:00:47,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:47,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132703 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:49,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:49,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133817 s, 76.5 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:00:51,159::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:00:51,160::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c11073c-d0d8-442e-8cc1-f798c2c4b596`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:00:51,191::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:51,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:51,201::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:51,202::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::INFO::2014-08-11 15:00:51,204::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:00:51,204::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c11073c-d0d8-442e-8cc1-f798c2c4b596`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:00:51,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c11073c-d0d8-442e-8cc1-f798c2c4b596`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:00:51,205::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:00:51,205::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:00:51,205::task::993::Storage.TaskManager.Task::(_decref) Task=`5c11073c-d0d8-442e-8cc1-f798c2c4b596`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:00:51,217::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:00:51,218::task::595::Storage.TaskManager.Task::(_updateState) Task=`73312307-84a2-4dd4-aa01-073f1b1aab49`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:00:51,218::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:00:51,219::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7a85e58d-e106-45c5-a1c9-6429cb458bea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:00:51,219::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:00:51,220::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:00:51,220::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7a85e58d-e106-45c5-a1c9-6429cb458bea`::Granted request >Thread-34::DEBUG::2014-08-11 15:00:51,220::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`73312307-84a2-4dd4-aa01-073f1b1aab49`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:00:51,221::task::993::Storage.TaskManager.Task::(_decref) Task=`73312307-84a2-4dd4-aa01-073f1b1aab49`::ref 1 aborting False >Thread-108::DEBUG::2014-08-11 15:00:51,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`afd5d15b-0f34-4c95-acce-24f5e6e81629`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:00:51,232::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:00:51,234::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419135', 'lastCheck': '4.2', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:00:51,235::task::1191::Storage.TaskManager.Task::(prepare) Task=`afd5d15b-0f34-4c95-acce-24f5e6e81629`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419135', 'lastCheck': '4.2', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:00:51,236::task::595::Storage.TaskManager.Task::(_updateState) Task=`afd5d15b-0f34-4c95-acce-24f5e6e81629`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:00:51,237::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:00:51,239::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:00:51,240::task::993::Storage.TaskManager.Task::(_decref) Task=`afd5d15b-0f34-4c95-acce-24f5e6e81629`::ref 0 aborting False >Thread-34::INFO::2014-08-11 15:00:51,263::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:51,263::task::1191::Storage.TaskManager.Task::(prepare) Task=`73312307-84a2-4dd4-aa01-073f1b1aab49`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244771840', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:00:51,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`73312307-84a2-4dd4-aa01-073f1b1aab49`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:00:51,264::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:00:51,264::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:00:51,264::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:00:51,265::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:00:51,265::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:00:51,265::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:00:51,265::task::993::Storage.TaskManager.Task::(_decref) Task=`73312307-84a2-4dd4-aa01-073f1b1aab49`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:00:51,269::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:00:51,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:51,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277003 s, 37.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:53,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:53,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013267 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:55,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:55,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132858 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:56,207::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:56,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:00:56,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:00:57,029::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:57,030::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:00:57,032::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:00:57,032::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:57,050::__init__::232::IOProcess::(_processLogs) (6157) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:57,051::__init__::232::IOProcess::(_processLogs) (6157) Queuing response >Thread-16::DEBUG::2014-08-11 15:00:57,052::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000495423 s, 918 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:00:57,053::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:57,053::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:57,054::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:57,054::__init__::232::IOProcess::(_processLogs) (6158) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:00:57,054::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|(6158) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:57,056::__init__::232::IOProcess::(_processLogs) (6146) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:57,057::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:57,057::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:57,058::__init__::232::IOProcess::(_processLogs) (6159) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:57,058::__init__::232::IOProcess::(_processLogs) (6159) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:57,058::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:57,059::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:57,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:57,060::__init__::232::IOProcess::(_processLogs) (6160) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:57,060::__init__::232::IOProcess::(_processLogs) (6160) Queuing response >Thread-14::DEBUG::2014-08-11 15:00:57,060::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:00:57,061::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:00:57,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:00:57,062::__init__::232::IOProcess::(_processLogs) (6161) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:00:57,063::__init__::232::IOProcess::(_processLogs) (6161) Queuing response >Dummy-33::DEBUG::2014-08-11 15:00:57,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:57,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131924 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:00:59,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:00:59,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131322 s, 78.0 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:01:01,285::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:01:01,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`39ea7fa7-65cb-46b7-98b4-dc9e999d8c34`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:01,287::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:01,296::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:01,298::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:01,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::INFO::2014-08-11 15:01:01,300::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:01,301::task::1191::Storage.TaskManager.Task::(prepare) Task=`39ea7fa7-65cb-46b7-98b4-dc9e999d8c34`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:01,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`39ea7fa7-65cb-46b7-98b4-dc9e999d8c34`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:01,303::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:01,304::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:01,304::task::993::Storage.TaskManager.Task::(_decref) Task=`39ea7fa7-65cb-46b7-98b4-dc9e999d8c34`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:01:01,316::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:01:01,317::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb38bf87-4122-47f9-8dab-5130abdbd52a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:01:01,317::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:01:01,318::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e800567d-66e5-4795-ab20-52f4b232ed08`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:01:01,318::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:01:01,319::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:01:01,319::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e800567d-66e5-4795-ab20-52f4b232ed08`::Granted request >Thread-34::DEBUG::2014-08-11 15:01:01,319::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`eb38bf87-4122-47f9-8dab-5130abdbd52a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:01:01,320::task::993::Storage.TaskManager.Task::(_decref) Task=`eb38bf87-4122-47f9-8dab-5130abdbd52a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:01:01,340::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244706304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:01,340::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb38bf87-4122-47f9-8dab-5130abdbd52a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244706304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:01,340::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb38bf87-4122-47f9-8dab-5130abdbd52a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:01:01,341::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:01:01,341::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:01:01,341::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:01:01,342::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:01:01,342::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:01:01,342::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:01:01,342::task::993::Storage.TaskManager.Task::(_decref) Task=`eb38bf87-4122-47f9-8dab-5130abdbd52a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:01,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:02,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0292805 s, 35.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:04,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:04,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133084 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:06,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:06,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132384 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:06,309::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:06,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:06,314::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:06,358::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdbbb213-a8e8-4dd5-8235-5f81f96af324`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:06,358::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:01:06,359::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000495423', 'lastCheck': '9.3', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:06,359::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdbbb213-a8e8-4dd5-8235-5f81f96af324`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000495423', 'lastCheck': '9.3', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:06,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdbbb213-a8e8-4dd5-8235-5f81f96af324`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:06,359::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:06,359::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:06,360::task::993::Storage.TaskManager.Task::(_decref) Task=`bdbbb213-a8e8-4dd5-8235-5f81f96af324`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:01:06,378::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:01:07,063::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:01:07,064::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:01:07,064::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:07,064::__init__::232::IOProcess::(_processLogs) (6162) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:01:07,066::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:01:07,084::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414733 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:07,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:07,086::__init__::232::IOProcess::(_processLogs) (6163) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:01:07,087::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:07,087::__init__::232::IOProcess::(_processLogs) (6164) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:07,088::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:07,088::__init__::232::IOProcess::(_processLogs) (6165) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:07,089::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:07,089::__init__::232::IOProcess::(_processLogs) (6166) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:07,090::__init__::232::IOProcess::(_processLogs) (6166) Queuing response >Dummy-33::DEBUG::2014-08-11 15:01:08,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:08,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132029 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:10,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:11,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:11,322::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:11,323::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:11,354::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:01:11,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf35f90-a7f0-4e55-86ea-c0cedd3a95c6`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:11,355::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:01:11,900::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:11,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`daf35f90-a7f0-4e55-86ea-c0cedd3a95c6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:11,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf35f90-a7f0-4e55-86ea-c0cedd3a95c6`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:11,903::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:11,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:11,904::task::993::Storage.TaskManager.Task::(_decref) Task=`daf35f90-a7f0-4e55-86ea-c0cedd3a95c6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:11,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.5191 s, 674 kB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 15:01:11,914::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:01:11,915::task::595::Storage.TaskManager.Task::(_updateState) Task=`568240bf-390d-48e5-97a5-1ef5ede26e08`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:01:11,916::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:01:11,916::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`db24cc4e-c105-4467-94c5-5ecfe9ced377`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:01:11,917::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:01:11,917::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:01:11,917::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`db24cc4e-c105-4467-94c5-5ecfe9ced377`::Granted request >Thread-34::DEBUG::2014-08-11 15:01:11,918::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`568240bf-390d-48e5-97a5-1ef5ede26e08`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:01:11,918::task::993::Storage.TaskManager.Task::(_decref) Task=`568240bf-390d-48e5-97a5-1ef5ede26e08`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:01:11,965::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244640768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:11,965::task::1191::Storage.TaskManager.Task::(prepare) Task=`568240bf-390d-48e5-97a5-1ef5ede26e08`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858244640768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:11,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`568240bf-390d-48e5-97a5-1ef5ede26e08`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:01:11,965::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:01:11,966::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:01:11,966::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:01:11,966::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:01:11,967::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:01:11,967::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:01:11,967::task::993::Storage.TaskManager.Task::(_decref) Task=`568240bf-390d-48e5-97a5-1ef5ede26e08`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:13,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:13,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132977 s, 77.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:01:14,789::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e56e08f-b0fd-4059-887f-f51adc63fbcd`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:01:14,789::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:01:14,790::__init__::232::IOProcess::(_processLogs) (6167) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:01:14,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:14,792::__init__::232::IOProcess::(_processLogs) (6168) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:01:14,793::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:01:14,793::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e56e08f-b0fd-4059-887f-f51adc63fbcd`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:01:14,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e56e08f-b0fd-4059-887f-f51adc63fbcd`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:01:14,794::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:01:14,794::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:01:14,794::task::993::Storage.TaskManager.Task::(_decref) Task=`8e56e08f-b0fd-4059-887f-f51adc63fbcd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:16,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:16,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137638 s, 74.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:16,330::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:16,331::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:16,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:01:17,092::__init__::232::IOProcess::(_processLogs) (6169) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:01:17,093::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:01:17,111::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420646 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:17,112::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:17,113::__init__::232::IOProcess::(_processLogs) (6170) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:01:17,114::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:17,114::__init__::232::IOProcess::(_processLogs) (6171) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:17,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:01:17,132::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:17,133::__init__::232::IOProcess::(_processLogs) (6172) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:17,133::__init__::232::IOProcess::(_processLogs) (6172) Queuing response >Thread-14::DEBUG::2014-08-11 15:01:17,134::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:01:17,134::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:17,134::__init__::232::IOProcess::(_processLogs) (6173) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:01:18,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:18,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135684 s, 75.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:20,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:20,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013288 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:21,339::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:21,340::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:21,341::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:21,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`711d435e-2f00-47d4-9a1a-72e4809ceaef`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:21,467::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:01:21,467::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420646', 'lastCheck': '4.3', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:21,467::task::1191::Storage.TaskManager.Task::(prepare) Task=`711d435e-2f00-47d4-9a1a-72e4809ceaef`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420646', 'lastCheck': '4.3', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:21,468::task::595::Storage.TaskManager.Task::(_updateState) Task=`711d435e-2f00-47d4-9a1a-72e4809ceaef`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:21,468::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:21,468::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:21,468::task::993::Storage.TaskManager.Task::(_decref) Task=`711d435e-2f00-47d4-9a1a-72e4809ceaef`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:01:21,489::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:21,981::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:01:21,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ec057db-11ec-4037-94fe-f9be6ae79c44`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:21,985::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:01:22,000::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:22,000::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ec057db-11ec-4037-94fe-f9be6ae79c44`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:22,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ec057db-11ec-4037-94fe-f9be6ae79c44`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:22,000::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:22,001::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:22,001::task::993::Storage.TaskManager.Task::(_decref) Task=`1ec057db-11ec-4037-94fe-f9be6ae79c44`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:01:22,012::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:01:22,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c711b2f-c6af-4e12-b443-2717b4586dcd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:01:22,013::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:01:22,014::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`715cdc76-998c-4eb1-869d-e8b64bbd8ec8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:01:22,014::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:01:22,014::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:01:22,014::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`715cdc76-998c-4eb1-869d-e8b64bbd8ec8`::Granted request >Thread-34::DEBUG::2014-08-11 15:01:22,015::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5c711b2f-c6af-4e12-b443-2717b4586dcd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:01:22,015::task::993::Storage.TaskManager.Task::(_decref) Task=`5c711b2f-c6af-4e12-b443-2717b4586dcd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:01:22,035::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858233499648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:22,035::task::1191::Storage.TaskManager.Task::(prepare) Task=`5c711b2f-c6af-4e12-b443-2717b4586dcd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858233499648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:22,036::task::595::Storage.TaskManager.Task::(_updateState) Task=`5c711b2f-c6af-4e12-b443-2717b4586dcd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:01:22,036::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:01:22,036::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:01:22,036::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:01:22,037::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:01:22,037::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:01:22,037::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:01:22,037::task::993::Storage.TaskManager.Task::(_decref) Task=`5c711b2f-c6af-4e12-b443-2717b4586dcd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:22,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:22,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166795 s, 61.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:24,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:24,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131526 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:26,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:26,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:26,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:26,360::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:01:26,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228512 s, 44.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:27,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:27,138::__init__::232::IOProcess::(_processLogs) (6174) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:01:27,139::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:01:27,158::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000411283 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:27,159::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:27,159::__init__::232::IOProcess::(_processLogs) (6175) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:01:27,161::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:27,161::__init__::232::IOProcess::(_processLogs) (6176) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:27,162::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:27,162::__init__::232::IOProcess::(_processLogs) (6177) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:27,163::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:27,163::__init__::232::IOProcess::(_processLogs) (6178) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:01:28,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:28,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132615 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:30,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:30,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132138 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:31,366::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:31,392::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:31,393::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:32,051::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:01:32,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`fae2fd34-4310-4cfb-bd99-1df2596f9e05`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:32,055::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:01:32,069::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:32,071::task::1191::Storage.TaskManager.Task::(prepare) Task=`fae2fd34-4310-4cfb-bd99-1df2596f9e05`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:32,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`fae2fd34-4310-4cfb-bd99-1df2596f9e05`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:32,074::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:32,075::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:32,076::task::993::Storage.TaskManager.Task::(_decref) Task=`fae2fd34-4310-4cfb-bd99-1df2596f9e05`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:01:32,093::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:01:32,096::task::595::Storage.TaskManager.Task::(_updateState) Task=`458a4cd7-1860-40eb-be19-be4ed484984a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:01:32,098::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:01:32,102::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`40fd90d8-228d-4d12-9248-48f0f069734f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:01:32,102::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:01:32,105::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:01:32,106::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`40fd90d8-228d-4d12-9248-48f0f069734f`::Granted request >Thread-34::DEBUG::2014-08-11 15:01:32,110::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`458a4cd7-1860-40eb-be19-be4ed484984a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:01:32,111::task::993::Storage.TaskManager.Task::(_decref) Task=`458a4cd7-1860-40eb-be19-be4ed484984a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:01:32,182::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858233499648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:32,183::task::1191::Storage.TaskManager.Task::(prepare) Task=`458a4cd7-1860-40eb-be19-be4ed484984a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858233499648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:32,185::task::595::Storage.TaskManager.Task::(_updateState) Task=`458a4cd7-1860-40eb-be19-be4ed484984a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:01:32,188::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:01:32,191::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:01:32,197::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:01:32,201::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:01:32,204::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:01:32,207::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:01:32,209::task::993::Storage.TaskManager.Task::(_decref) Task=`458a4cd7-1860-40eb-be19-be4ed484984a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:32,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:32,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0316045 s, 32.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:34,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:34,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013131 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:36,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:36,493::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:36,494::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:36,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8976c44-ed53-4221-91fe-eb198db7884e`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:36,586::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:01:36,587::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411283', 'lastCheck': '9.4', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:36,587::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8976c44-ed53-4221-91fe-eb198db7884e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000411283', 'lastCheck': '9.4', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:36,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8976c44-ed53-4221-91fe-eb198db7884e`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:36,587::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:36,588::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:36,588::task::993::Storage.TaskManager.Task::(_decref) Task=`c8976c44-ed53-4221-91fe-eb198db7884e`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:01:36,610::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:01:36,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:36,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132106 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:37,165::__init__::232::IOProcess::(_processLogs) (6179) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:01:37,166::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:01:37,185::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0004791 s, 950 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:37,186::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:37,186::__init__::232::IOProcess::(_processLogs) (6180) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:01:37,188::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:37,188::__init__::232::IOProcess::(_processLogs) (6181) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:37,189::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:37,189::__init__::232::IOProcess::(_processLogs) (6182) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:37,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:37,190::__init__::232::IOProcess::(_processLogs) (6183) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:01:38,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:38,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164681 s, 62.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:41,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:41,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132401 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:41,501::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:41,502::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:41,503::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:42,225::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:01:42,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`18b41f8d-590b-4d80-8e46-d3dace3916d8`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:42,226::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:01:42,239::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:42,239::task::1191::Storage.TaskManager.Task::(prepare) Task=`18b41f8d-590b-4d80-8e46-d3dace3916d8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:42,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`18b41f8d-590b-4d80-8e46-d3dace3916d8`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:42,240::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:42,240::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:42,240::task::993::Storage.TaskManager.Task::(_decref) Task=`18b41f8d-590b-4d80-8e46-d3dace3916d8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:01:42,252::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:01:42,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`f77fd102-8176-4500-8521-977d6252cc3a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:01:42,253::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:01:42,254::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d643a869-ae22-4b2e-b31c-8e6989cac819`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:01:42,254::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:01:42,255::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:01:42,255::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d643a869-ae22-4b2e-b31c-8e6989cac819`::Granted request >Thread-34::DEBUG::2014-08-11 15:01:42,256::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f77fd102-8176-4500-8521-977d6252cc3a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:01:42,256::task::993::Storage.TaskManager.Task::(_decref) Task=`f77fd102-8176-4500-8521-977d6252cc3a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:01:42,275::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858225242112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:42,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`f77fd102-8176-4500-8521-977d6252cc3a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858225242112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:42,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`f77fd102-8176-4500-8521-977d6252cc3a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:01:42,276::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:01:42,277::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:01:42,277::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:01:42,277::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:01:42,277::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:01:42,278::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:01:42,278::task::993::Storage.TaskManager.Task::(_decref) Task=`f77fd102-8176-4500-8521-977d6252cc3a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:43,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:43,094::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131408 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:45,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:45,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131356 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:46,510::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:46,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:46,593::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:01:47,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 15:01:47,202::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:47,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0333675 s, 30.7 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:01:47,227::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000605248 s, 752 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:47,230::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:01:47,231::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:47,232::__init__::232::IOProcess::(_processLogs) (6185) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:01:47,236::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:01:47,238::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:47,240::__init__::232::IOProcess::(_processLogs) (6186) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:47,241::__init__::232::IOProcess::(_processLogs) (6186) Queuing response >Thread-14::DEBUG::2014-08-11 15:01:47,242::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:01:47,244::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:47,245::__init__::232::IOProcess::(_processLogs) (6187) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:47,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:01:47,249::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:01:47,250::__init__::232::IOProcess::(_processLogs) (6188) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:01:49,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:49,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133433 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:51,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:51,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133573 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:51,599::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:51,601::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:51,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:51,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`b17ecb05-8f44-46a3-a7f0-05685b7e01af`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:51,711::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:01:51,712::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000605248', 'lastCheck': '4.5', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:51,712::task::1191::Storage.TaskManager.Task::(prepare) Task=`b17ecb05-8f44-46a3-a7f0-05685b7e01af`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000605248', 'lastCheck': '4.5', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:01:51,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`b17ecb05-8f44-46a3-a7f0-05685b7e01af`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:51,713::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:51,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:51,713::task::993::Storage.TaskManager.Task::(_decref) Task=`b17ecb05-8f44-46a3-a7f0-05685b7e01af`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:01:51,735::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:01:52,290::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:01:52,291::task::595::Storage.TaskManager.Task::(_updateState) Task=`b30cd102-25b3-415d-bd7b-56f341477eb6`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:01:52,291::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:01:52,304::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:52,304::task::1191::Storage.TaskManager.Task::(prepare) Task=`b30cd102-25b3-415d-bd7b-56f341477eb6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:01:52,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`b30cd102-25b3-415d-bd7b-56f341477eb6`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:01:52,304::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:01:52,305::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:01:52,305::task::993::Storage.TaskManager.Task::(_decref) Task=`b30cd102-25b3-415d-bd7b-56f341477eb6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:01:52,317::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:01:52,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`bbabc7d9-5581-4c05-8a51-0cea8feebc0f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:01:52,318::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:01:52,319::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`651505b9-4cf2-4f5c-ab71-f441cfafa036`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:01:52,319::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:01:52,320::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:01:52,320::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`651505b9-4cf2-4f5c-ab71-f441cfafa036`::Granted request >Thread-34::DEBUG::2014-08-11 15:01:52,320::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bbabc7d9-5581-4c05-8a51-0cea8feebc0f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:01:52,321::task::993::Storage.TaskManager.Task::(_decref) Task=`bbabc7d9-5581-4c05-8a51-0cea8feebc0f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:01:52,341::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858224259072', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:52,341::task::1191::Storage.TaskManager.Task::(prepare) Task=`bbabc7d9-5581-4c05-8a51-0cea8feebc0f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858224259072', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:01:52,342::task::595::Storage.TaskManager.Task::(_updateState) Task=`bbabc7d9-5581-4c05-8a51-0cea8feebc0f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:01:52,342::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:01:52,342::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:01:52,342::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:01:52,343::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:01:52,343::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:01:52,343::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:01:52,343::task::993::Storage.TaskManager.Task::(_decref) Task=`bbabc7d9-5581-4c05-8a51-0cea8feebc0f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:01:53,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:53,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.025673 s, 39.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:55,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:55,481::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132581 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:56,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:56,609::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:01:56,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:01:57,254::__init__::232::IOProcess::(_processLogs) (6189) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:01:57,255::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:01:57,273::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0004217 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:01:57,275::__init__::232::IOProcess::(_processLogs) (6190) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:01:57,276::__init__::232::IOProcess::(_processLogs) (6191) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:57,277::__init__::232::IOProcess::(_processLogs) (6192) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:01:57,278::__init__::232::IOProcess::(_processLogs) (6193) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:01:57,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:57,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131314 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:01:59,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:01:59,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131536 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:01,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:01,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171863 s, 59.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:01,701::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:01,706::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:01,711::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:02,357::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:02:02,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`30ef3f46-77b2-41ec-8a94-1f5d56da92ec`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:02,361::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:02:02,375::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:02,377::task::1191::Storage.TaskManager.Task::(prepare) Task=`30ef3f46-77b2-41ec-8a94-1f5d56da92ec`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:02,378::task::595::Storage.TaskManager.Task::(_updateState) Task=`30ef3f46-77b2-41ec-8a94-1f5d56da92ec`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:02,379::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:02,380::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:02,381::task::993::Storage.TaskManager.Task::(_decref) Task=`30ef3f46-77b2-41ec-8a94-1f5d56da92ec`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:02:02,395::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:02:02,398::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8fd2da5-21bf-4bb2-9cda-4e5a4a72fedc`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:02:02,400::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:02:02,403::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fb2ced35-e13e-4038-9dcb-3671987481bb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:02:02,404::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:02:02,406::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:02:02,407::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fb2ced35-e13e-4038-9dcb-3671987481bb`::Granted request >Thread-34::DEBUG::2014-08-11 15:02:02,411::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b8fd2da5-21bf-4bb2-9cda-4e5a4a72fedc`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:02:02,413::task::993::Storage.TaskManager.Task::(_decref) Task=`b8fd2da5-21bf-4bb2-9cda-4e5a4a72fedc`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:02:02,470::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858223276032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:02,472::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8fd2da5-21bf-4bb2-9cda-4e5a4a72fedc`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858223276032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:02,474::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8fd2da5-21bf-4bb2-9cda-4e5a4a72fedc`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:02:02,475::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:02:02,476::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:02:02,478::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:02:02,480::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:02:02,482::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:02:02,483::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:02:02,486::task::993::Storage.TaskManager.Task::(_decref) Task=`b8fd2da5-21bf-4bb2-9cda-4e5a4a72fedc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:02:03,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:03,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131791 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:05,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:05,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131413 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:06,721::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:06,723::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:06,792::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:06,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`343c52f5-503e-4a0d-aee0-c70ad18219a7`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:06,835::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:02:06,835::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0004217', 'lastCheck': '9.6', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:02:06,835::task::1191::Storage.TaskManager.Task::(prepare) Task=`343c52f5-503e-4a0d-aee0-c70ad18219a7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0004217', 'lastCheck': '9.6', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:02:06,836::task::595::Storage.TaskManager.Task::(_updateState) Task=`343c52f5-503e-4a0d-aee0-c70ad18219a7`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:06,836::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:06,836::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:06,836::task::993::Storage.TaskManager.Task::(_decref) Task=`343c52f5-503e-4a0d-aee0-c70ad18219a7`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:02:06,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:02:07,280::__init__::232::IOProcess::(_processLogs) (6194) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:02:07,281::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:02:07,299::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412993 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:02:07,301::__init__::232::IOProcess::(_processLogs) (6195) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:07,302::__init__::232::IOProcess::(_processLogs) (6196) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:07,303::__init__::232::IOProcess::(_processLogs) (6197) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:07,304::__init__::232::IOProcess::(_processLogs) (6198) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:02:07,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:07,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131991 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:09,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:09,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133046 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:11,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:11,892::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:11,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:02:11,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:11,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131315 s, 78.0 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:02:12,502::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:02:12,503::task::595::Storage.TaskManager.Task::(_updateState) Task=`a084ab8a-bb88-4312-b253-1261c98d4205`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:12,503::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:02:12,517::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:12,517::task::1191::Storage.TaskManager.Task::(prepare) Task=`a084ab8a-bb88-4312-b253-1261c98d4205`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:12,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`a084ab8a-bb88-4312-b253-1261c98d4205`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:12,518::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:12,518::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:12,518::task::993::Storage.TaskManager.Task::(_decref) Task=`a084ab8a-bb88-4312-b253-1261c98d4205`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:02:12,528::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:02:12,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`ddafc95f-796f-441f-adcd-323ca98edf38`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:02:12,529::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:02:12,530::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b773011d-0525-4ce4-a56f-b5dd2a609ff1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:02:12,530::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:02:12,530::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:02:12,531::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b773011d-0525-4ce4-a56f-b5dd2a609ff1`::Granted request >Thread-34::DEBUG::2014-08-11 15:02:12,531::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ddafc95f-796f-441f-adcd-323ca98edf38`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:02:12,531::task::993::Storage.TaskManager.Task::(_decref) Task=`ddafc95f-796f-441f-adcd-323ca98edf38`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:02:12,551::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858222882816', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:12,552::task::1191::Storage.TaskManager.Task::(prepare) Task=`ddafc95f-796f-441f-adcd-323ca98edf38`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858222882816', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:12,552::task::595::Storage.TaskManager.Task::(_updateState) Task=`ddafc95f-796f-441f-adcd-323ca98edf38`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:02:12,552::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:02:12,552::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:02:12,553::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:02:12,553::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:02:12,553::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:02:12,553::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:02:12,554::task::993::Storage.TaskManager.Task::(_decref) Task=`ddafc95f-796f-441f-adcd-323ca98edf38`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:02:14,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:14,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132867 s, 77.1 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:02:14,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdc3b807-92d0-4644-9342-e9854444c33d`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:02:14,902::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:02:14,903::__init__::232::IOProcess::(_processLogs) (6199) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:02:14,906::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:14,906::__init__::232::IOProcess::(_processLogs) (6200) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:02:14,906::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:02:14,907::task::1191::Storage.TaskManager.Task::(prepare) Task=`fdc3b807-92d0-4644-9342-e9854444c33d`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:02:14,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdc3b807-92d0-4644-9342-e9854444c33d`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:02:14,907::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:02:14,907::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:02:14,908::task::993::Storage.TaskManager.Task::(_decref) Task=`fdc3b807-92d0-4644-9342-e9854444c33d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:02:16,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:16,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131418 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:16,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:16,902::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:16,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:02:17,306::__init__::232::IOProcess::(_processLogs) (6201) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:02:17,307::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:02:17,326::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426043 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:02:17,328::__init__::232::IOProcess::(_processLogs) (6202) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:17,329::__init__::232::IOProcess::(_processLogs) (6203) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:17,330::__init__::232::IOProcess::(_processLogs) (6204) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:17,331::__init__::232::IOProcess::(_processLogs) (6205) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:02:18,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:18,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132102 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:20,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:20,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132898 s, 77.1 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:02:21,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`4541db9a-6d38-4ae9-aa56-9dde42890b6f`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:21,967::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:02:21,992::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426043', 'lastCheck': '4.7', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:02:21,992::task::1191::Storage.TaskManager.Task::(prepare) Task=`4541db9a-6d38-4ae9-aa56-9dde42890b6f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426043', 'lastCheck': '4.7', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:02:21,992::task::595::Storage.TaskManager.Task::(_updateState) Task=`4541db9a-6d38-4ae9-aa56-9dde42890b6f`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:21,992::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:21,992::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:21,993::task::993::Storage.TaskManager.Task::(_decref) Task=`4541db9a-6d38-4ae9-aa56-9dde42890b6f`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:21,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:22,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:22,001::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:22,012::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:02:22,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:22,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131953 s, 77.6 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:02:22,567::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:02:22,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b65901c-3eba-4ec6-9c22-4ccd47050116`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:22,569::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:02:22,581::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:22,582::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b65901c-3eba-4ec6-9c22-4ccd47050116`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:22,582::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b65901c-3eba-4ec6-9c22-4ccd47050116`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:22,582::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:22,582::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:22,583::task::993::Storage.TaskManager.Task::(_decref) Task=`3b65901c-3eba-4ec6-9c22-4ccd47050116`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:02:22,595::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:02:22,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebcfd4ee-7c44-40e1-a593-aa522a99f9cf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:02:22,596::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:02:22,596::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`77da52b9-34a8-494c-bcf5-f644ad0c38f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:02:22,597::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:02:22,597::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:02:22,597::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`77da52b9-34a8-494c-bcf5-f644ad0c38f9`::Granted request >Thread-34::DEBUG::2014-08-11 15:02:22,598::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ebcfd4ee-7c44-40e1-a593-aa522a99f9cf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:02:22,598::task::993::Storage.TaskManager.Task::(_decref) Task=`ebcfd4ee-7c44-40e1-a593-aa522a99f9cf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:02:22,618::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858220392448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:22,618::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebcfd4ee-7c44-40e1-a593-aa522a99f9cf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858220392448', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:22,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebcfd4ee-7c44-40e1-a593-aa522a99f9cf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:02:22,619::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:02:22,619::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:02:22,619::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:02:22,620::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:02:22,620::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:02:22,620::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:02:22,620::task::993::Storage.TaskManager.Task::(_decref) Task=`ebcfd4ee-7c44-40e1-a593-aa522a99f9cf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:02:24,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:24,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131553 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:26,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:26,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133351 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:27,008::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:27,009::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:27,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:02:27,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:27,333::__init__::232::IOProcess::(_processLogs) (6206) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:02:27,334::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:02:27,353::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416225 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:02:27,355::__init__::232::IOProcess::(_processLogs) (6207) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:27,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:27,356::__init__::232::IOProcess::(_processLogs) (6208) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:27,357::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6208) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:27,357::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:27,358::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:27,358::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:27,358::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6209) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:27,359::__init__::232::IOProcess::(_processLogs) (6209) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:27,359::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:27,360::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:27,360::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:27,360::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6210) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:27,360::__init__::232::IOProcess::(_processLogs) (6210) Queuing response >Dummy-33::DEBUG::2014-08-11 15:02:28,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:28,437::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132981 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:30,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:30,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133014 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:32,101::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:32,106::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:32,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:02:32,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:32,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280008 s, 36.6 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:02:32,639::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:02:32,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b7a7a61-6f1f-409e-8489-e3af2a92d166`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:32,643::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:02:32,659::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:32,661::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b7a7a61-6f1f-409e-8489-e3af2a92d166`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:32,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b7a7a61-6f1f-409e-8489-e3af2a92d166`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:32,663::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:32,664::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:32,666::task::993::Storage.TaskManager.Task::(_decref) Task=`2b7a7a61-6f1f-409e-8489-e3af2a92d166`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:02:32,681::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:02:32,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`a25122a6-5b6e-4d64-a37c-24629420cba6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:02:32,685::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:02:32,690::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2e08115b-c066-458f-a8b0-0b5065894d57`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:02:32,693::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:02:32,697::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:02:32,698::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2e08115b-c066-458f-a8b0-0b5065894d57`::Granted request >Thread-34::DEBUG::2014-08-11 15:02:32,701::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a25122a6-5b6e-4d64-a37c-24629420cba6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:02:32,703::task::993::Storage.TaskManager.Task::(_decref) Task=`a25122a6-5b6e-4d64-a37c-24629420cba6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:02:32,756::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858219671552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:32,758::task::1191::Storage.TaskManager.Task::(prepare) Task=`a25122a6-5b6e-4d64-a37c-24629420cba6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858219671552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:32,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`a25122a6-5b6e-4d64-a37c-24629420cba6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:02:32,760::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:02:32,761::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:02:32,763::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:02:32,765::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:02:32,766::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:02:32,767::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:02:32,769::task::993::Storage.TaskManager.Task::(_decref) Task=`a25122a6-5b6e-4d64-a37c-24629420cba6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:02:34,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:34,892::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133801 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:36,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:36,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131219 s, 78.0 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:02:37,107::task::595::Storage.TaskManager.Task::(_updateState) Task=`0acff86a-9ce2-43ab-a4d9-3d22478ae8d2`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:37,107::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:02:37,192::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416225', 'lastCheck': '9.8', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:02:37,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`0acff86a-9ce2-43ab-a4d9-3d22478ae8d2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416225', 'lastCheck': '9.8', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:02:37,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`0acff86a-9ce2-43ab-a4d9-3d22478ae8d2`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:37,192::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:37,192::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:37,193::task::993::Storage.TaskManager.Task::(_decref) Task=`0acff86a-9ce2-43ab-a4d9-3d22478ae8d2`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:37,206::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:37,207::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:37,208::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:37,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:02:37,362::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:02:37,363::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,364::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,364::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,364::__init__::232::IOProcess::(_processLogs) (6211) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:02:37,365::__init__::232::IOProcess::(_processLogs) (6211) Queuing response >Thread-16::DEBUG::2014-08-11 15:02:37,372::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:02:37,373::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,373::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,374::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,374::__init__::232::IOProcess::(_processLogs) (6212) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:02:37,375::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6212) Queuing response >Thread-16::DEBUG::2014-08-11 15:02:37,375::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:02:37,376::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,376::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,377::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,377::__init__::232::IOProcess::(_processLogs) (6213) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:02:37,378::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6213) Queuing response >Thread-16::DEBUG::2014-08-11 15:02:37,378::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:02:37,379::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,380::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,380::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,380::__init__::232::IOProcess::(_processLogs) (6214) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:37,381::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6214) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:37,381::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,382::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,382::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,382::__init__::232::IOProcess::(_processLogs) (6215) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:02:37,383::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 15:02:37,383::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6215) Queuing response >Thread-16::INFO::2014-08-11 15:02:37,384::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:02:37,384::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:02:37,385::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,385::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,386::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,386::__init__::232::IOProcess::(_processLogs) (6216) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:37,387::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6216) Queuing response >Thread-16::DEBUG::2014-08-11 15:02:37,388::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:02:37,406::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042182 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:02:37,407::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,407::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,407::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,408::__init__::232::IOProcess::(_processLogs) (6217) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:37,408::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6217) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:37,409::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,409::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,410::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,410::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6218) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:37,410::__init__::232::IOProcess::(_processLogs) (6218) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:37,411::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,411::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,412::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,412::__init__::232::IOProcess::(_processLogs) (6219) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:37,412::__init__::232::IOProcess::(_processLogs) (6219) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:37,413::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:37,413::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:37,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:37,414::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6220) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:37,414::__init__::232::IOProcess::(_processLogs) (6220) Queuing response >Dummy-33::DEBUG::2014-08-11 15:02:38,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:39,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133002 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:41,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:41,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132332 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:42,215::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:42,216::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:42,292::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:42,788::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:02:42,789::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f384b82-82b8-4ec6-a6f3-f64730cc9cec`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:42,789::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:02:42,802::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:42,802::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f384b82-82b8-4ec6-a6f3-f64730cc9cec`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:42,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f384b82-82b8-4ec6-a6f3-f64730cc9cec`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:42,802::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:42,803::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:42,803::task::993::Storage.TaskManager.Task::(_decref) Task=`3f384b82-82b8-4ec6-a6f3-f64730cc9cec`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:02:42,813::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:02:42,814::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c7b6235-e664-4495-bf72-228ce753a597`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:02:42,814::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:02:42,815::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f7cb5bd-0286-408d-a2a7-c69f34ca5bcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:02:42,815::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:02:42,815::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:02:42,816::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f7cb5bd-0286-408d-a2a7-c69f34ca5bcf`::Granted request >Thread-34::DEBUG::2014-08-11 15:02:42,816::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7c7b6235-e664-4495-bf72-228ce753a597`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:02:42,816::task::993::Storage.TaskManager.Task::(_decref) Task=`7c7b6235-e664-4495-bf72-228ce753a597`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:02:42,836::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858159116288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:42,836::task::1191::Storage.TaskManager.Task::(prepare) Task=`7c7b6235-e664-4495-bf72-228ce753a597`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858159116288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:42,837::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c7b6235-e664-4495-bf72-228ce753a597`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:02:42,837::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:02:42,837::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:02:42,837::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:02:42,838::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:02:42,838::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:02:42,838::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:02:42,839::task::993::Storage.TaskManager.Task::(_decref) Task=`7c7b6235-e664-4495-bf72-228ce753a597`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:02:43,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:43,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132755 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:45,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:45,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131997 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:47,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:47,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013271 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:47,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:47,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:47,302::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:02:47,416::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:47,416::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:47,417::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:47,417::__init__::232::IOProcess::(_processLogs) (6221) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:47,418::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6221) Queuing response >Thread-16::DEBUG::2014-08-11 15:02:47,419::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:02:47,438::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421775 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:02:47,439::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:47,439::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:47,440::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:47,440::__init__::232::IOProcess::(_processLogs) (6222) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:47,440::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6222) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:47,441::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:47,442::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:47,442::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:47,442::__init__::232::IOProcess::(_processLogs) (6223) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:47,442::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6223) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:47,443::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:47,444::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:47,444::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:47,444::__init__::232::IOProcess::(_processLogs) (6224) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:47,444::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6224) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:47,445::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:47,445::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:47,446::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:47,446::__init__::232::IOProcess::(_processLogs) (6225) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:47,446::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6225) Queuing response >Dummy-33::DEBUG::2014-08-11 15:02:49,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:49,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132979 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:51,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:51,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133172 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:52,310::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:52,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e8c31c4-3887-47a7-b748-61e517b61e2b`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:52,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::INFO::2014-08-11 15:02:52,392::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:02:52,394::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421775', 'lastCheck': '4.9', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:02:52,396::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e8c31c4-3887-47a7-b748-61e517b61e2b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421775', 'lastCheck': '4.9', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:52,397::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:52,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e8c31c4-3887-47a7-b748-61e517b61e2b`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:52,399::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:52,400::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:52,401::task::993::Storage.TaskManager.Task::(_decref) Task=`0e8c31c4-3887-47a7-b748-61e517b61e2b`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:02:52,422::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:02:52,853::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:02:52,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`52b25e2e-de64-456e-bd1c-2193034d6a9a`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:02:52,854::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:02:52,866::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:52,867::task::1191::Storage.TaskManager.Task::(prepare) Task=`52b25e2e-de64-456e-bd1c-2193034d6a9a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:02:52,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`52b25e2e-de64-456e-bd1c-2193034d6a9a`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:02:52,867::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:02:52,867::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:02:52,867::task::993::Storage.TaskManager.Task::(_decref) Task=`52b25e2e-de64-456e-bd1c-2193034d6a9a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:02:52,878::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:02:52,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f692f2b-86f5-44a6-896c-fe141c996f80`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:02:52,879::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:02:52,880::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9924204-33df-40da-bce5-fdd5a7e10d5c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:02:52,880::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:02:52,881::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:02:52,881::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a9924204-33df-40da-bce5-fdd5a7e10d5c`::Granted request >Thread-34::DEBUG::2014-08-11 15:02:52,882::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5f692f2b-86f5-44a6-896c-fe141c996f80`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:02:52,882::task::993::Storage.TaskManager.Task::(_decref) Task=`5f692f2b-86f5-44a6-896c-fe141c996f80`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:02:52,901::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:52,902::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f692f2b-86f5-44a6-896c-fe141c996f80`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:02:52,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f692f2b-86f5-44a6-896c-fe141c996f80`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:02:52,902::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:02:52,903::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:02:52,903::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:02:52,903::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:02:52,903::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:02:52,904::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:02:52,904::task::993::Storage.TaskManager.Task::(_decref) Task=`5f692f2b-86f5-44a6-896c-fe141c996f80`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:02:53,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:53,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0429078 s, 23.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:55,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:55,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147768 s, 69.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:57,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:57,403::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:02:57,448::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:57,495::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:02:57,496::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:02:57,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:02:57,516::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:57,518::__init__::232::IOProcess::(_processLogs) (6226) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:57,519::__init__::232::IOProcess::(_processLogs) (6226) Queuing response >Thread-16::DEBUG::2014-08-11 15:02:57,521::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000774078 s, 588 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:02:57,523::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:57,523::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:57,523::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:57,524::__init__::232::IOProcess::(_processLogs) (6227) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:02:57,524::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6227) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:57,525::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:57,525::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:57,526::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:57,526::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6228) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:57,526::__init__::232::IOProcess::(_processLogs) (6228) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:57,527::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:57,527::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:57,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:57,528::__init__::232::IOProcess::(_processLogs) (6229) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:57,528::__init__::232::IOProcess::(_processLogs) (6229) Queuing response >Thread-14::DEBUG::2014-08-11 15:02:57,529::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:02:57,529::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:02:57,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:02:57,529::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6230) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:02:57,530::__init__::232::IOProcess::(_processLogs) (6230) Queuing response >Dummy-33::DEBUG::2014-08-11 15:02:57,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:57,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131798 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:02:59,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:02:59,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133375 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:01,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:01,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134837 s, 75.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:02,530::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:02,535::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:02,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:02,921::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:03:02,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`09a052c4-4e68-403d-9011-659fbf438bb0`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:02,925::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:03:02,939::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:02,940::task::1191::Storage.TaskManager.Task::(prepare) Task=`09a052c4-4e68-403d-9011-659fbf438bb0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:02,940::task::595::Storage.TaskManager.Task::(_updateState) Task=`09a052c4-4e68-403d-9011-659fbf438bb0`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:02,941::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:02,941::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:02,941::task::993::Storage.TaskManager.Task::(_decref) Task=`09a052c4-4e68-403d-9011-659fbf438bb0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:03:02,951::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:03:02,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ebe9079-c543-4ea6-b77e-01fa84a17cd4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:03:02,952::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:03:02,953::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`86f1364a-36f0-483e-9e4d-edb3b2b0767c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:03:02,953::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:03:02,953::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:03:02,954::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`86f1364a-36f0-483e-9e4d-edb3b2b0767c`::Granted request >Thread-34::DEBUG::2014-08-11 15:03:02,954::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8ebe9079-c543-4ea6-b77e-01fa84a17cd4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:03:02,954::task::993::Storage.TaskManager.Task::(_decref) Task=`8ebe9079-c543-4ea6-b77e-01fa84a17cd4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:03:02,974::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:02,975::task::1191::Storage.TaskManager.Task::(prepare) Task=`8ebe9079-c543-4ea6-b77e-01fa84a17cd4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:02,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ebe9079-c543-4ea6-b77e-01fa84a17cd4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:03:02,975::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:03:02,975::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:03:02,976::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:03:02,976::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:03:02,976::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:03:02,976::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:03:02,977::task::993::Storage.TaskManager.Task::(_decref) Task=`8ebe9079-c543-4ea6-b77e-01fa84a17cd4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:03:03,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:03,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132713 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:05,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:05,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132174 s, 77.5 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:03:07,519::task::595::Storage.TaskManager.Task::(_updateState) Task=`5b8dece1-9975-406a-b141-efd31e84007d`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:07,519::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:03:07,519::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000774078', 'lastCheck': '10.0', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:07,520::task::1191::Storage.TaskManager.Task::(prepare) Task=`5b8dece1-9975-406a-b141-efd31e84007d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000774078', 'lastCheck': '10.0', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:07,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`5b8dece1-9975-406a-b141-efd31e84007d`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:07,520::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:07,520::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:07,520::task::993::Storage.TaskManager.Task::(_decref) Task=`5b8dece1-9975-406a-b141-efd31e84007d`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:03:07,532::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:07,532::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:07,532::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:07,533::__init__::232::IOProcess::(_processLogs) (6231) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:07,533::__init__::232::IOProcess::(_processLogs) (6231) Queuing response >Thread-16::DEBUG::2014-08-11 15:03:07,533::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-108::DEBUG::2014-08-11 15:03:07,553::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:03:07,557::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00427021 s, 107 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:07,593::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:07,594::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:07,594::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:07,594::__init__::232::IOProcess::(_processLogs) (6232) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:07,594::__init__::232::IOProcess::(_processLogs) (6232) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:07,595::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:07,596::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:07,597::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:07,597::__init__::232::IOProcess::(_processLogs) (6233) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:07,597::__init__::232::IOProcess::(_processLogs) (6233) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:07,597::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:07,598::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:07,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:07,599::__init__::232::IOProcess::(_processLogs) (6234) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:07,599::__init__::232::IOProcess::(_processLogs) (6234) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:07,600::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:07,600::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:07,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:07,602::__init__::232::IOProcess::(_processLogs) (6235) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:07,602::__init__::232::IOProcess::(_processLogs) (6235) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:07,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:07,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:07,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:03:07,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:08,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013269 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:10,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:10,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137468 s, 74.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:12,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:12,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131789 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:12,615::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:12,616::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:12,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:12,989::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:03:12,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`b262c0c3-ebe5-4c8e-8927-85a23305ea79`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:12,990::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:03:13,004::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:13,004::task::1191::Storage.TaskManager.Task::(prepare) Task=`b262c0c3-ebe5-4c8e-8927-85a23305ea79`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:13,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`b262c0c3-ebe5-4c8e-8927-85a23305ea79`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:13,004::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:13,005::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:13,005::task::993::Storage.TaskManager.Task::(_decref) Task=`b262c0c3-ebe5-4c8e-8927-85a23305ea79`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:03:13,016::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:03:13,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`38ec7898-cdf5-4c02-a771-01ecc74bbd5d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:03:13,017::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:03:13,017::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d9a5deb2-455a-4ce8-9119-4d1f7fcbf1d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:03:13,018::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:03:13,018::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:03:13,018::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d9a5deb2-455a-4ce8-9119-4d1f7fcbf1d9`::Granted request >Thread-34::DEBUG::2014-08-11 15:03:13,019::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`38ec7898-cdf5-4c02-a771-01ecc74bbd5d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:03:13,019::task::993::Storage.TaskManager.Task::(_decref) Task=`38ec7898-cdf5-4c02-a771-01ecc74bbd5d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:03:13,040::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:13,040::task::1191::Storage.TaskManager.Task::(prepare) Task=`38ec7898-cdf5-4c02-a771-01ecc74bbd5d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:13,040::task::595::Storage.TaskManager.Task::(_updateState) Task=`38ec7898-cdf5-4c02-a771-01ecc74bbd5d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:03:13,041::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:03:13,041::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:03:13,041::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:03:13,042::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:03:13,042::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:03:13,042::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:03:13,042::task::993::Storage.TaskManager.Task::(_decref) Task=`38ec7898-cdf5-4c02-a771-01ecc74bbd5d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:03:14,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:14,191::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132082 s, 77.5 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:03:15,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`839ff807-d591-41a7-9387-0e9528e4a68d`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:03:15,025::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:03:15,026::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:15,026::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:15,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:15,027::__init__::232::IOProcess::(_processLogs) (6236) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:03:15,028::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6236) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:15,029::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:15,030::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 15:03:15,030::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:03:15,031::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:03:15,031::task::1191::Storage.TaskManager.Task::(prepare) Task=`839ff807-d591-41a7-9387-0e9528e4a68d`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:03:15,032::__init__::232::IOProcess::(_processLogs) (6237) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:03:15,033::task::595::Storage.TaskManager.Task::(_updateState) Task=`839ff807-d591-41a7-9387-0e9528e4a68d`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:03:15,034::__init__::232::IOProcess::(_processLogs) (6237) Queuing response >Thread-55::DEBUG::2014-08-11 15:03:15,034::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:03:15,035::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:03:15,035::task::993::Storage.TaskManager.Task::(_decref) Task=`839ff807-d591-41a7-9387-0e9528e4a68d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:03:16,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:16,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132171 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:17,603::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:17,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:17,604::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:03:17,605::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:03:17,605::__init__::232::IOProcess::(_processLogs) (6238) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:17,623::__init__::232::IOProcess::(_processLogs) (6238) Queuing response >Thread-16::DEBUG::2014-08-11 15:03:17,625::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00043688 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:17,693::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:17,694::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:17,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:17,697::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:17,699::__init__::232::IOProcess::(_processLogs) (6239) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:17,700::__init__::232::IOProcess::(_processLogs) (6239) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:17,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:17,703::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:17,704::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:17,706::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:17,707::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:17,708::__init__::232::IOProcess::(_processLogs) (6240) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:17,710::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6240) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:17,711::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:17,712::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:17,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:17,712::__init__::232::IOProcess::(_processLogs) (6241) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:17,712::__init__::232::IOProcess::(_processLogs) (6241) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:17,713::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:17,713::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:17,713::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:17,714::__init__::232::IOProcess::(_processLogs) (6242) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:17,714::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6242) Queuing response >Dummy-33::DEBUG::2014-08-11 15:03:18,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:18,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134036 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:20,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:20,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133349 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:22,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:22,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131153 s, 78.1 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:03:22,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ca3a934-bda1-421b-a1ff-7798751766d1`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:22,655::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:03:22,693::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043688', 'lastCheck': '5.0', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:22,693::task::1191::Storage.TaskManager.Task::(prepare) Task=`6ca3a934-bda1-421b-a1ff-7798751766d1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043688', 'lastCheck': '5.0', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:22,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ca3a934-bda1-421b-a1ff-7798751766d1`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:22,693::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:22,693::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:22,694::task::993::Storage.TaskManager.Task::(_decref) Task=`6ca3a934-bda1-421b-a1ff-7798751766d1`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:22,713::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:22,714::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:22,719::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:22,720::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:23,053::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:03:23,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`bca5309e-fe17-421e-bedb-bc0fac15b45a`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:23,054::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:03:23,067::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:23,067::task::1191::Storage.TaskManager.Task::(prepare) Task=`bca5309e-fe17-421e-bedb-bc0fac15b45a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:23,068::task::595::Storage.TaskManager.Task::(_updateState) Task=`bca5309e-fe17-421e-bedb-bc0fac15b45a`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:23,068::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:23,068::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:23,068::task::993::Storage.TaskManager.Task::(_decref) Task=`bca5309e-fe17-421e-bedb-bc0fac15b45a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:03:23,078::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:03:23,079::task::595::Storage.TaskManager.Task::(_updateState) Task=`87b21d98-4ca7-42f1-8b3b-529870a742de`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:03:23,079::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:03:23,080::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`59d3dd34-6ab8-4140-a38f-a468170cc4fd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:03:23,080::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:03:23,080::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:03:23,081::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`59d3dd34-6ab8-4140-a38f-a468170cc4fd`::Granted request >Thread-34::DEBUG::2014-08-11 15:03:23,081::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`87b21d98-4ca7-42f1-8b3b-529870a742de`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:03:23,081::task::993::Storage.TaskManager.Task::(_decref) Task=`87b21d98-4ca7-42f1-8b3b-529870a742de`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:03:23,102::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:23,103::task::1191::Storage.TaskManager.Task::(prepare) Task=`87b21d98-4ca7-42f1-8b3b-529870a742de`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157871104', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:23,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`87b21d98-4ca7-42f1-8b3b-529870a742de`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:03:23,103::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:03:23,103::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:03:23,104::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:03:23,104::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:03:23,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:03:23,104::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:03:23,105::task::993::Storage.TaskManager.Task::(_decref) Task=`87b21d98-4ca7-42f1-8b3b-529870a742de`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:03:24,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:24,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130854 s, 78.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:26,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:26,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132508 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:27,716::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:27,718::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:03:27,718::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:27,730::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:27,794::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:27,814::__init__::232::IOProcess::(_processLogs) (6243) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:27,815::__init__::232::IOProcess::(_processLogs) (6243) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:27,818::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:27,820::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:03:27,821::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0011525 s, 395 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:27,823::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:27,823::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:27,824::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:27,824::__init__::232::IOProcess::(_processLogs) (6244) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:27,826::__init__::232::IOProcess::(_processLogs) (6244) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:27,826::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:27,826::__init__::232::IOProcess::(_processLogs) (6245) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:27,827::__init__::232::IOProcess::(_processLogs) (6245) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:27,828::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:27,828::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:27,828::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:27,829::__init__::232::IOProcess::(_processLogs) (6246) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:27,829::__init__::232::IOProcess::(_processLogs) DEBUG|(6246) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:27,830::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:27,830::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:27,831::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:27,832::__init__::232::IOProcess::(_processLogs) (6247) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:27,832::__init__::232::IOProcess::(_processLogs) (6247) Queuing response >Dummy-33::DEBUG::2014-08-11 15:03:28,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:28,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132393 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:30,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:30,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132791 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:32,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:32,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277561 s, 36.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:32,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:32,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:32,850::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:33,131::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:03:33,132::task::595::Storage.TaskManager.Task::(_updateState) Task=`75f85aca-2eb0-4620-8b8f-06cd8493bd6c`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:33,132::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:03:33,145::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:33,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`75f85aca-2eb0-4620-8b8f-06cd8493bd6c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:33,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`75f85aca-2eb0-4620-8b8f-06cd8493bd6c`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:33,146::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:33,146::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:33,146::task::993::Storage.TaskManager.Task::(_decref) Task=`75f85aca-2eb0-4620-8b8f-06cd8493bd6c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:03:33,159::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:03:33,160::task::595::Storage.TaskManager.Task::(_updateState) Task=`b18cce27-bce1-43bd-9e83-0b2c9becd4f6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:03:33,160::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:03:33,161::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6a4dc012-0557-4966-aa1c-1cb537d003d7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:03:33,161::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:03:33,163::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:03:33,164::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6a4dc012-0557-4966-aa1c-1cb537d003d7`::Granted request >Thread-34::DEBUG::2014-08-11 15:03:33,164::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b18cce27-bce1-43bd-9e83-0b2c9becd4f6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:03:33,164::task::993::Storage.TaskManager.Task::(_decref) Task=`b18cce27-bce1-43bd-9e83-0b2c9becd4f6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:03:33,184::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:33,184::task::1191::Storage.TaskManager.Task::(prepare) Task=`b18cce27-bce1-43bd-9e83-0b2c9becd4f6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:33,185::task::595::Storage.TaskManager.Task::(_updateState) Task=`b18cce27-bce1-43bd-9e83-0b2c9becd4f6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:03:33,185::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:03:33,185::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:03:33,185::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:03:33,186::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:03:33,186::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:03:33,186::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:03:33,187::task::993::Storage.TaskManager.Task::(_decref) Task=`b18cce27-bce1-43bd-9e83-0b2c9becd4f6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:03:34,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:34,962::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131976 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:36,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:37,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136317 s, 75.1 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:03:37,827::task::595::Storage.TaskManager.Task::(_updateState) Task=`21004d7b-389e-4fa0-bca2-439c20a69eee`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:37,828::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:03:37,828::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0011525', 'lastCheck': '10.0', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:37,828::task::1191::Storage.TaskManager.Task::(prepare) Task=`21004d7b-389e-4fa0-bca2-439c20a69eee`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0011525', 'lastCheck': '10.0', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:37,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`21004d7b-389e-4fa0-bca2-439c20a69eee`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:37,829::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:37,829::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:37,829::task::993::Storage.TaskManager.Task::(_decref) Task=`21004d7b-389e-4fa0-bca2-439c20a69eee`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:03:37,834::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:37,836::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108::DEBUG::2014-08-11 15:03:37,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:37,863::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:37,895::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:37,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:37,899::__init__::232::IOProcess::(_processLogs) (6248) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:37,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:37,906::__init__::232::IOProcess::(_processLogs) DEBUG|(6248) Queuing response >Thread-16::DEBUG::2014-08-11 15:03:37,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:03:37,931::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00108563 s, 419 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:37,933::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:37,933::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:37,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:37,934::__init__::232::IOProcess::(_processLogs) (6249) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:37,934::__init__::232::IOProcess::(_processLogs) (6249) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:37,935::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:37,935::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:37,936::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:37,936::__init__::232::IOProcess::(_processLogs) (6250) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:37,937::__init__::232::IOProcess::(_processLogs) (6250) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:37,937::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:37,938::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:37,938::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:37,939::__init__::232::IOProcess::(_processLogs) (6251) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:37,939::__init__::232::IOProcess::(_processLogs) (6251) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:37,939::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:37,940::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:37,940::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:37,941::__init__::232::IOProcess::(_processLogs) (6252) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:37,942::__init__::232::IOProcess::(_processLogs) (6252) Queuing response >Dummy-33::DEBUG::2014-08-11 15:03:39,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:39,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132263 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:41,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:41,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133446 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:42,918::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:42,919::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:42,994::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:03:43,152::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:43,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132358 s, 77.4 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:03:43,212::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:03:43,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3dfa7bd-d69f-441d-bfc8-631cdf94c3d5`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:43,213::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:03:43,226::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:43,226::task::1191::Storage.TaskManager.Task::(prepare) Task=`c3dfa7bd-d69f-441d-bfc8-631cdf94c3d5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:43,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3dfa7bd-d69f-441d-bfc8-631cdf94c3d5`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:43,227::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:43,227::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:43,227::task::993::Storage.TaskManager.Task::(_decref) Task=`c3dfa7bd-d69f-441d-bfc8-631cdf94c3d5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:03:43,237::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:03:43,238::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbd5271f-7cb4-49de-922c-d3d99ee06813`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:03:43,238::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:03:43,239::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`00364aa5-7797-4816-b140-87042a36b68f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:03:43,239::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:03:43,240::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:03:43,240::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`00364aa5-7797-4816-b140-87042a36b68f`::Granted request >Thread-34::DEBUG::2014-08-11 15:03:43,241::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cbd5271f-7cb4-49de-922c-d3d99ee06813`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:03:43,241::task::993::Storage.TaskManager.Task::(_decref) Task=`cbd5271f-7cb4-49de-922c-d3d99ee06813`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:03:43,261::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:43,261::task::1191::Storage.TaskManager.Task::(prepare) Task=`cbd5271f-7cb4-49de-922c-d3d99ee06813`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:43,261::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbd5271f-7cb4-49de-922c-d3d99ee06813`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:03:43,262::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:03:43,262::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:03:43,262::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:03:43,263::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:03:43,263::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:03:43,263::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:03:43,263::task::993::Storage.TaskManager.Task::(_decref) Task=`cbd5271f-7cb4-49de-922c-d3d99ee06813`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:03:45,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:45,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132169 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:47,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:47,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013147 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:47,942::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:47,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:03:47,944::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:03:47,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:48,013::__init__::232::IOProcess::(_processLogs) (6253) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:48,015::__init__::232::IOProcess::(_processLogs) (6253) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:48,015::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:03:48,017::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000570791 s, 797 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:48,020::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:48,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:48,022::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:03:48,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:48,024::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:48,025::__init__::232::IOProcess::(_processLogs) (6254) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:48,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6254) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:48,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:48,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:48,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:48,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6255) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:48,031::__init__::232::IOProcess::(_processLogs) (6255) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:48,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:48,032::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:48,032::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:48,032::__init__::232::IOProcess::(_processLogs) (6256) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:48,033::__init__::232::IOProcess::(_processLogs) (6256) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:48,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:48,034::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:48,034::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:48,035::__init__::232::IOProcess::(_processLogs) (6257) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:48,035::__init__::232::IOProcess::(_processLogs) (6257) Queuing response >Dummy-33::DEBUG::2014-08-11 15:03:49,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:49,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133523 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:51,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:51,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01327 s, 77.2 MB/s\n'; <rc> = 0 >Thread-108::DEBUG::2014-08-11 15:03:53,029::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfb9a366-2cb6-4a3a-9b77-0e0bd72f2f97`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:53,029::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:53,034::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::INFO::2014-08-11 15:03:53,095::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000570791', 'lastCheck': '5.1', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:53,096::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfb9a366-2cb6-4a3a-9b77-0e0bd72f2f97`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000570791', 'lastCheck': '5.1', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:03:53,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfb9a366-2cb6-4a3a-9b77-0e0bd72f2f97`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:53,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:53,099::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:53,100::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:53,102::task::993::Storage.TaskManager.Task::(_decref) Task=`bfb9a366-2cb6-4a3a-9b77-0e0bd72f2f97`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:53,104::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:53,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:03:53,276::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:03:53,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`886f4214-c529-4da9-9510-269a766634ad`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:03:53,277::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:03:53,289::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:53,290::task::1191::Storage.TaskManager.Task::(prepare) Task=`886f4214-c529-4da9-9510-269a766634ad`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:03:53,290::task::595::Storage.TaskManager.Task::(_updateState) Task=`886f4214-c529-4da9-9510-269a766634ad`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:03:53,290::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:03:53,291::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:03:53,291::task::993::Storage.TaskManager.Task::(_decref) Task=`886f4214-c529-4da9-9510-269a766634ad`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:03:53,301::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:03:53,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f80d486-f70b-4812-8b68-78005ff02df7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:03:53,302::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:03:53,303::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`def2b753-8692-49c1-9dcd-931b60381e8c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:03:53,303::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:03:53,304::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:03:53,304::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`def2b753-8692-49c1-9dcd-931b60381e8c`::Granted request >Thread-34::DEBUG::2014-08-11 15:03:53,304::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5f80d486-f70b-4812-8b68-78005ff02df7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:03:53,305::task::993::Storage.TaskManager.Task::(_decref) Task=`5f80d486-f70b-4812-8b68-78005ff02df7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:03:53,324::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:53,325::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f80d486-f70b-4812-8b68-78005ff02df7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:03:53,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f80d486-f70b-4812-8b68-78005ff02df7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:03:53,325::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:03:53,325::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:03:53,326::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:03:53,326::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:03:53,326::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:03:53,326::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:03:53,327::task::993::Storage.TaskManager.Task::(_decref) Task=`5f80d486-f70b-4812-8b68-78005ff02df7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:03:53,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:53,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131297 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:55,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:55,515::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132397 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:03:57,536::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:57,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131882 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:58,036::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:58,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:03:58,038::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:03:58,094::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:58,112::__init__::232::IOProcess::(_processLogs) (6258) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:58,113::__init__::232::IOProcess::(_processLogs) (6258) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:58,114::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:58,116::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:03:58,117::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:03:58,118::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000941243 s, 483 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:03:58,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:58,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:58,119::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:58,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6259) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:03:58,121::__init__::232::IOProcess::(_processLogs) (6259) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:58,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:58,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:58,123::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:58,123::__init__::232::IOProcess::(_processLogs) (6260) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:58,123::__init__::232::IOProcess::(_processLogs) (6260) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:58,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:58,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:58,125::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:58,125::__init__::232::IOProcess::(_processLogs) (6261) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:58,125::__init__::232::IOProcess::(_processLogs) (6261) Queuing response >Thread-14::DEBUG::2014-08-11 15:03:58,126::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:03:58,126::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:03:58,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:03:58,128::__init__::232::IOProcess::(_processLogs) (6262) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:03:58,128::__init__::232::IOProcess::(_processLogs) (6262) Queuing response >Dummy-33::DEBUG::2014-08-11 15:03:59,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:03:59,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0151232 s, 67.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:01,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:01,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133706 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:03,124::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:03,125::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:03,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:03,339::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-108::DEBUG::2014-08-11 15:04:03,339::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1ddd8cb-eb4d-4299-a63c-f2765fc66c0f`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:03,340::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:04:03,353::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:03,353::task::1191::Storage.TaskManager.Task::(prepare) Task=`d1ddd8cb-eb4d-4299-a63c-f2765fc66c0f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:03,353::task::595::Storage.TaskManager.Task::(_updateState) Task=`d1ddd8cb-eb4d-4299-a63c-f2765fc66c0f`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:03,354::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:03,354::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:03,354::task::993::Storage.TaskManager.Task::(_decref) Task=`d1ddd8cb-eb4d-4299-a63c-f2765fc66c0f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:04:03,364::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 15:04:03,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b3477e0-4b20-4e0e-af3d-996d352d9bbe`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:04:03,365::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:04:03,366::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ffa83bf1-dea8-48c6-988a-edb4e37a0f78`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:04:03,366::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:04:03,367::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:04:03,367::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ffa83bf1-dea8-48c6-988a-edb4e37a0f78`::Granted request >Thread-34::DEBUG::2014-08-11 15:04:03,367::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9b3477e0-4b20-4e0e-af3d-996d352d9bbe`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:04:03,368::task::993::Storage.TaskManager.Task::(_decref) Task=`9b3477e0-4b20-4e0e-af3d-996d352d9bbe`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:04:03,387::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:03,388::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b3477e0-4b20-4e0e-af3d-996d352d9bbe`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:03,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b3477e0-4b20-4e0e-af3d-996d352d9bbe`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:04:03,388::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:04:03,388::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:04:03,389::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:04:03,389::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:04:03,389::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:04:03,389::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:04:03,390::task::993::Storage.TaskManager.Task::(_decref) Task=`9b3477e0-4b20-4e0e-af3d-996d352d9bbe`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:04:03,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:03,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131218 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:05,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:05,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131545 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:07,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:07,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131923 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:08,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:08,131::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:04:08,132::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:08,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:08,196::__init__::232::IOProcess::(_processLogs) (6263) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:08,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6263) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:08,198::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:04:08,200::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:08,202::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:08,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`74cd27ae-ed08-4025-998d-d954878447ad`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:08,224::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:04:08,224::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000941243', 'lastCheck': '10.1', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:04:08,224::task::1191::Storage.TaskManager.Task::(prepare) Task=`74cd27ae-ed08-4025-998d-d954878447ad`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000941243', 'lastCheck': '10.1', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:04:08,224::task::595::Storage.TaskManager.Task::(_updateState) Task=`74cd27ae-ed08-4025-998d-d954878447ad`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:08,225::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:08,225::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:08,225::task::993::Storage.TaskManager.Task::(_decref) Task=`74cd27ae-ed08-4025-998d-d954878447ad`::ref 0 aborting False >Thread-16::DEBUG::2014-08-11 15:04:08,229::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000574072 s, 793 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:08,232::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:08,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:08,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:08,236::__init__::232::IOProcess::(_processLogs) (6264) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:08,238::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6264) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:08,241::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:08,242::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:08,243::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:08,244::__init__::232::IOProcess::(_processLogs) (6265) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:08,245::__init__::232::IOProcess::(_processLogs) (6265) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:08,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:08,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:08,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:08,249::__init__::232::IOProcess::(_processLogs) (6266) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:08,249::__init__::232::IOProcess::(_processLogs) (6266) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:08,249::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:08,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:08,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:08,250::__init__::232::IOProcess::(_processLogs) (6267) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:08,251::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6267) Queuing response >Thread-108::DEBUG::2014-08-11 15:04:08,262::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:04:09,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:09,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132941 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:11,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:11,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132337 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:13,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:13,211::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:13,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:13,402::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:04:13,403::task::595::Storage.TaskManager.Task::(_updateState) Task=`d46d6f21-755a-4b85-aa77-3a8df5764b2a`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:13,404::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:04:13,417::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:13,417::task::1191::Storage.TaskManager.Task::(prepare) Task=`d46d6f21-755a-4b85-aa77-3a8df5764b2a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:13,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`d46d6f21-755a-4b85-aa77-3a8df5764b2a`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:13,418::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:13,418::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:13,418::task::993::Storage.TaskManager.Task::(_decref) Task=`d46d6f21-755a-4b85-aa77-3a8df5764b2a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:04:13,428::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:04:13,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8722f14-5288-4e50-ac2b-34b054bdd527`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:04:13,429::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:04:13,430::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c51522ad-ceb7-4708-aef3-3e7503706968`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:04:13,430::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:04:13,431::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:04:13,431::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c51522ad-ceb7-4708-aef3-3e7503706968`::Granted request >Thread-34::DEBUG::2014-08-11 15:04:13,431::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a8722f14-5288-4e50-ac2b-34b054bdd527`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:04:13,432::task::993::Storage.TaskManager.Task::(_decref) Task=`a8722f14-5288-4e50-ac2b-34b054bdd527`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:04:13,452::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:13,452::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8722f14-5288-4e50-ac2b-34b054bdd527`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:13,452::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8722f14-5288-4e50-ac2b-34b054bdd527`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:04:13,452::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:04:13,453::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:04:13,453::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:04:13,453::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:04:13,453::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:04:13,454::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:04:13,454::task::993::Storage.TaskManager.Task::(_decref) Task=`a8722f14-5288-4e50-ac2b-34b054bdd527`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:04:13,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:14,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132422 s, 77.3 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:04:15,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`758e145f-7859-4a9a-9612-66a8b0cf8f38`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:04:15,150::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:04:15,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:15,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:15,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:15,151::__init__::232::IOProcess::(_processLogs) (6268) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:04:15,152::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6268) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:15,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:15,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:15,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:15,154::__init__::232::IOProcess::(_processLogs) (6269) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:04:15,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6269) Queuing response >Thread-55::INFO::2014-08-11 15:04:15,155::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:04:15,155::task::1191::Storage.TaskManager.Task::(prepare) Task=`758e145f-7859-4a9a-9612-66a8b0cf8f38`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:04:15,155::task::595::Storage.TaskManager.Task::(_updateState) Task=`758e145f-7859-4a9a-9612-66a8b0cf8f38`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:04:15,156::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:04:15,156::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:04:15,156::task::993::Storage.TaskManager.Task::(_decref) Task=`758e145f-7859-4a9a-9612-66a8b0cf8f38`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:04:16,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:16,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134169 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:18,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:18,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132625 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:18,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:18,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:18,255::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:18,255::__init__::232::IOProcess::(_processLogs) (6270) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:18,255::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6270) Queuing response >Thread-16::DEBUG::2014-08-11 15:04:18,256::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:18,312::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:18,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:18,314::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:04:18,316::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000363272 s, 1.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:18,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:18,318::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:18,318::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:18,318::__init__::232::IOProcess::(_processLogs) (6271) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:18,318::__init__::232::IOProcess::(_processLogs) (6271) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:18,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:18,320::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:18,320::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:18,320::__init__::232::IOProcess::(_processLogs) (6272) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:18,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6272) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:18,322::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:18,322::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:18,323::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:18,323::__init__::232::IOProcess::(_processLogs) (6273) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:18,323::__init__::232::IOProcess::(_processLogs) (6273) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:18,324::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:18,324::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:18,324::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:18,324::__init__::232::IOProcess::(_processLogs) (6274) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:18,325::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6274) Queuing response >Dummy-33::DEBUG::2014-08-11 15:04:20,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:20,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135088 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:22,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:22,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131965 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:23,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:23,322::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:23,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`28a8710f-63cf-4c62-9d8c-712166ffed24`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:23,397::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:04:23,399::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000363272', 'lastCheck': '5.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:23,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:23,401::task::1191::Storage.TaskManager.Task::(prepare) Task=`28a8710f-63cf-4c62-9d8c-712166ffed24`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000363272', 'lastCheck': '5.1', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:04:23,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`28a8710f-63cf-4c62-9d8c-712166ffed24`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:23,404::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:23,404::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:23,404::task::993::Storage.TaskManager.Task::(_decref) Task=`28a8710f-63cf-4c62-9d8c-712166ffed24`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:04:23,423::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:23,467::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:04:23,468::task::595::Storage.TaskManager.Task::(_updateState) Task=`69e48109-b021-417d-b524-e33248a43623`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:23,468::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:04:23,481::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:23,481::task::1191::Storage.TaskManager.Task::(prepare) Task=`69e48109-b021-417d-b524-e33248a43623`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:23,482::task::595::Storage.TaskManager.Task::(_updateState) Task=`69e48109-b021-417d-b524-e33248a43623`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:23,482::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:23,482::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:23,482::task::993::Storage.TaskManager.Task::(_decref) Task=`69e48109-b021-417d-b524-e33248a43623`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:04:23,492::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:04:23,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`220d216f-3881-4ebe-a80c-2620c24b3bb4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:04:23,493::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:04:23,494::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`334db2a6-c094-4039-b0ed-018bd7341ddc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:04:23,494::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:04:23,495::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:04:23,495::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`334db2a6-c094-4039-b0ed-018bd7341ddc`::Granted request >Thread-34::DEBUG::2014-08-11 15:04:23,495::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`220d216f-3881-4ebe-a80c-2620c24b3bb4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:04:23,496::task::993::Storage.TaskManager.Task::(_decref) Task=`220d216f-3881-4ebe-a80c-2620c24b3bb4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:04:23,515::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:23,516::task::1191::Storage.TaskManager.Task::(prepare) Task=`220d216f-3881-4ebe-a80c-2620c24b3bb4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:23,516::task::595::Storage.TaskManager.Task::(_updateState) Task=`220d216f-3881-4ebe-a80c-2620c24b3bb4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:04:23,516::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:04:23,516::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:04:23,517::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:04:23,517::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:04:23,517::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:04:23,518::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:04:23,518::task::993::Storage.TaskManager.Task::(_decref) Task=`220d216f-3881-4ebe-a80c-2620c24b3bb4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:04:24,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:24,305::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131868 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:26,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:26,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132989 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:28,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:28,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 15:04:28,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:04:28,397::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:04:28,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:04:28,418::__init__::232::IOProcess::(_processLogs) (6275) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:28,440::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:04:28,440::__init__::232::IOProcess::(_processLogs) (6275) Queuing response >Thread-16::DEBUG::2014-08-11 15:04:28,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000858403 s, 530 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:28,447::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:04:28,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:28,455::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:04:28,456::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:28,460::__init__::232::IOProcess::(_processLogs) (6276) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:28,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:28,464::__init__::232::IOProcess::(_processLogs) (6277) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:04:28,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0498049 s, 20.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:28,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:28,471::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:28,472::__init__::232::IOProcess::(_processLogs) (6278) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:28,476::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:28,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:28,478::__init__::232::IOProcess::(_processLogs) (6279) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:28,479::__init__::232::IOProcess::(_processLogs) (6279) Queuing response >Dummy-33::DEBUG::2014-08-11 15:04:30,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:30,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133034 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:32,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:32,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0286467 s, 35.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:33,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:33,467::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:33,496::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:33,530::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:04:33,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`36912b15-0988-4c4a-9ded-7598e0ee6c93`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:33,532::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-108::INFO::2014-08-11 15:04:33,544::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:33,545::task::1191::Storage.TaskManager.Task::(prepare) Task=`36912b15-0988-4c4a-9ded-7598e0ee6c93`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:33,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`36912b15-0988-4c4a-9ded-7598e0ee6c93`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:33,545::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:33,545::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:33,546::task::993::Storage.TaskManager.Task::(_decref) Task=`36912b15-0988-4c4a-9ded-7598e0ee6c93`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:04:33,556::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:04:33,556::task::595::Storage.TaskManager.Task::(_updateState) Task=`3671bdd0-0a0e-4362-b07b-d38d5e55b345`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:04:33,556::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:04:33,557::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7ffb171-38b0-45da-ad0d-f850b264f25b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:04:33,557::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:04:33,558::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:04:33,558::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7ffb171-38b0-45da-ad0d-f850b264f25b`::Granted request >Thread-34::DEBUG::2014-08-11 15:04:33,559::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3671bdd0-0a0e-4362-b07b-d38d5e55b345`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:04:33,559::task::993::Storage.TaskManager.Task::(_decref) Task=`3671bdd0-0a0e-4362-b07b-d38d5e55b345`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:04:33,578::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:33,579::task::1191::Storage.TaskManager.Task::(prepare) Task=`3671bdd0-0a0e-4362-b07b-d38d5e55b345`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:33,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`3671bdd0-0a0e-4362-b07b-d38d5e55b345`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:04:33,579::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:04:33,579::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:04:33,580::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:04:33,580::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:04:33,580::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:04:33,581::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:04:33,581::task::993::Storage.TaskManager.Task::(_decref) Task=`3671bdd0-0a0e-4362-b07b-d38d5e55b345`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:04:34,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:34,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131881 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:36,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:36,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131559 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:38,479::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:38,479::__init__::232::IOProcess::(_processLogs) (6280) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:04:38,496::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:38,515::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:38,516::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:04:38,519::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000409055 s, 1.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:38,520::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:04:38,523::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:38,525::__init__::232::IOProcess::(_processLogs) (6281) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:38,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108::DEBUG::2014-08-11 15:04:38,530::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8e62b92-5602-48ff-8c6f-c242123af8bf`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:04:38,530::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-108::INFO::2014-08-11 15:04:38,531::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-14::DEBUG::2014-08-11 15:04:38,532::__init__::232::IOProcess::(_processLogs) (6282) Got request for method 'access' >Thread-108::INFO::2014-08-11 15:04:38,533::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000858403', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 15:04:38,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108::DEBUG::2014-08-11 15:04:38,535::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8e62b92-5602-48ff-8c6f-c242123af8bf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000858403', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 15:04:38,536::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-108::DEBUG::2014-08-11 15:04:38,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8e62b92-5602-48ff-8c6f-c242123af8bf`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:38,538::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:04:38,538::__init__::232::IOProcess::(_processLogs) (6283) Got request for method 'access' >Thread-108::DEBUG::2014-08-11 15:04:38,538::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:38,540::task::993::Storage.TaskManager.Task::(_decref) Task=`e8e62b92-5602-48ff-8c6f-c242123af8bf`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:04:38,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:38,544::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:38,546::__init__::232::IOProcess::(_processLogs) (6284) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:38,547::__init__::232::IOProcess::(_processLogs) (6284) Queuing response >Thread-108::DEBUG::2014-08-11 15:04:38,566::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:04:38,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:38,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132483 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:40,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:41,002::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132687 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:43,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:43,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131839 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:43,531::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:43,532::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:43,594::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-108::DEBUG::2014-08-11 15:04:43,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7c9030c-78de-433a-85f3-7a10aa56b0cf`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:43,599::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:43,601::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::INFO::2014-08-11 15:04:43,613::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:43,613::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7c9030c-78de-433a-85f3-7a10aa56b0cf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-108::DEBUG::2014-08-11 15:04:43,614::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7c9030c-78de-433a-85f3-7a10aa56b0cf`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:43,614::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:43,614::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:43,614::task::993::Storage.TaskManager.Task::(_decref) Task=`c7c9030c-78de-433a-85f3-7a10aa56b0cf`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:04:43,624::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:04:43,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`7062a358-03b9-4ff4-b726-da72802d2b33`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:04:43,625::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:04:43,626::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2469ffcb-a4c3-4358-bcb1-63a96eac8f65`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:04:43,626::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:04:43,626::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:04:43,626::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2469ffcb-a4c3-4358-bcb1-63a96eac8f65`::Granted request >Thread-34::DEBUG::2014-08-11 15:04:43,627::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7062a358-03b9-4ff4-b726-da72802d2b33`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:04:43,627::task::993::Storage.TaskManager.Task::(_decref) Task=`7062a358-03b9-4ff4-b726-da72802d2b33`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:04:43,647::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:43,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`7062a358-03b9-4ff4-b726-da72802d2b33`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157805568', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:43,648::task::595::Storage.TaskManager.Task::(_updateState) Task=`7062a358-03b9-4ff4-b726-da72802d2b33`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:04:43,648::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:04:43,648::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:04:43,648::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:04:43,649::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:04:43,649::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:04:43,649::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:04:43,649::task::993::Storage.TaskManager.Task::(_decref) Task=`7062a358-03b9-4ff4-b726-da72802d2b33`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:04:45,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:45,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132303 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:47,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:47,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132531 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:48,544::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:48,544::__init__::232::IOProcess::(_processLogs) (6285) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:48,595::__init__::232::IOProcess::(_processLogs) (6285) Queuing response >Thread-16::DEBUG::2014-08-11 15:04:48,596::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:48,615::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:48,616::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:48,617::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:04:48,619::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00146384 s, 311 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:48,620::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:48,620::__init__::232::IOProcess::(_processLogs) (6286) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:48,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:48,622::__init__::232::IOProcess::(_processLogs) (6287) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:48,623::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:48,623::__init__::232::IOProcess::(_processLogs) (6288) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:48,623::__init__::232::IOProcess::(_processLogs) (6288) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:48,624::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:48,625::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:48,625::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:48,626::__init__::232::IOProcess::(_processLogs) (6289) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:48,627::__init__::232::IOProcess::(_processLogs) (6289) Queuing response >Dummy-33::DEBUG::2014-08-11 15:04:49,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:49,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131402 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:51,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:51,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131638 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:04:53,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:53,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131322 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:53,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:53,621::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:53,663::task::595::Storage.TaskManager.Task::(_updateState) Task=`bee6a0cf-4f14-4c8d-a19c-f0cbef3f4c7a`::moving from state init -> state preparing >Thread-108::INFO::2014-08-11 15:04:53,697::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-108::INFO::2014-08-11 15:04:53,699::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00146384', 'lastCheck': '5.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:53,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-108::DEBUG::2014-08-11 15:04:53,701::task::1191::Storage.TaskManager.Task::(prepare) Task=`bee6a0cf-4f14-4c8d-a19c-f0cbef3f4c7a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00146384', 'lastCheck': '5.1', 'valid': True}} >Thread-108::DEBUG::2014-08-11 15:04:53,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`bee6a0cf-4f14-4c8d-a19c-f0cbef3f4c7a`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-08-11 15:04:53,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-08-11 15:04:53,704::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-08-11 15:04:53,704::task::993::Storage.TaskManager.Task::(_decref) Task=`bee6a0cf-4f14-4c8d-a19c-f0cbef3f4c7a`::ref 0 aborting False >Thread-108::DEBUG::2014-08-11 15:04:53,722::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:04:55,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:55,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137751 s, 74.3 MB/s\n'; <rc> = 0 >Detector thread::DEBUG::2014-08-11 15:04:55,884::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:55534 >Detector thread::DEBUG::2014-08-11 15:04:55,894::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:55534 >Detector thread::DEBUG::2014-08-11 15:04:55,894::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:55534 >Detector thread::DEBUG::2014-08-11 15:04:55,895::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 55534) >Thread-110::DEBUG::2014-08-11 15:04:55,898::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-110::DEBUG::2014-08-11 15:04:55,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`76251047-4d1a-4a0f-b1a1-784a0d2d2161`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:04:55,899::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:04:55,912::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:04:55,912::task::1191::Storage.TaskManager.Task::(prepare) Task=`76251047-4d1a-4a0f-b1a1-784a0d2d2161`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:04:55,912::task::595::Storage.TaskManager.Task::(_updateState) Task=`76251047-4d1a-4a0f-b1a1-784a0d2d2161`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:04:55,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:04:55,913::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:04:55,913::task::993::Storage.TaskManager.Task::(_decref) Task=`76251047-4d1a-4a0f-b1a1-784a0d2d2161`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:04:55,923::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-34::DEBUG::2014-08-11 15:04:55,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`23bcd539-8d4e-48af-a37a-f6a502352574`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:04:55,924::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:04:55,925::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d333effb-c186-4364-8144-90153a390637`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:04:55,925::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:04:55,926::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:04:55,926::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d333effb-c186-4364-8144-90153a390637`::Granted request >Thread-34::DEBUG::2014-08-11 15:04:55,927::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`23bcd539-8d4e-48af-a37a-f6a502352574`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:04:55,927::task::993::Storage.TaskManager.Task::(_decref) Task=`23bcd539-8d4e-48af-a37a-f6a502352574`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:04:55,947::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157740032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:55,947::task::1191::Storage.TaskManager.Task::(prepare) Task=`23bcd539-8d4e-48af-a37a-f6a502352574`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157740032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:04:55,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`23bcd539-8d4e-48af-a37a-f6a502352574`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:04:55,948::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:04:55,948::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:04:55,948::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:04:55,948::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:04:55,949::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:04:55,949::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:04:55,949::task::993::Storage.TaskManager.Task::(_decref) Task=`23bcd539-8d4e-48af-a37a-f6a502352574`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:04:57,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:57,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132082 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:58,627::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:58,628::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:58,695::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:58,696::__init__::232::IOProcess::(_processLogs) (6290) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:58,696::__init__::232::IOProcess::(_processLogs) (6290) Queuing response >Thread-16::DEBUG::2014-08-11 15:04:58,697::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:58,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:58,717::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:04:58,718::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:04:58,720::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00163824 s, 278 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:04:58,721::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:58,721::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:58,722::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:58,722::__init__::232::IOProcess::(_processLogs) (6291) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:04:58,723::__init__::232::IOProcess::(_processLogs) DEBUG|(6291) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:58,724::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:58,725::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:58,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:58,725::__init__::232::IOProcess::(_processLogs) (6292) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:58,726::__init__::232::IOProcess::(_processLogs) (6292) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:58,726::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:58,727::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:58,727::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:58,728::__init__::232::IOProcess::(_processLogs) (6293) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:58,728::__init__::232::IOProcess::(_processLogs) (6293) Queuing response >Thread-14::DEBUG::2014-08-11 15:04:58,728::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:04:58,729::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:04:58,729::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:04:58,730::__init__::232::IOProcess::(_processLogs) (6294) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:04:58,731::__init__::232::IOProcess::(_processLogs) (6294) Queuing response >Dummy-33::DEBUG::2014-08-11 15:04:59,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:04:59,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158143 s, 64.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:01,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:01,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133016 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:03,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:03,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013209 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:03,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:03,727::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:03,796::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:05,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:05,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131234 s, 78.0 MB/s\n'; <rc> = 0 >Thread-110::DEBUG::2014-08-11 15:05:05,961::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-110::DEBUG::2014-08-11 15:05:05,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`92d53951-87b7-434b-a0b8-9ca6c5ff4897`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:05,963::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:05:05,976::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:05,976::task::1191::Storage.TaskManager.Task::(prepare) Task=`92d53951-87b7-434b-a0b8-9ca6c5ff4897`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:05,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`92d53951-87b7-434b-a0b8-9ca6c5ff4897`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:05,976::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:05,977::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:05,977::task::993::Storage.TaskManager.Task::(_decref) Task=`92d53951-87b7-434b-a0b8-9ca6c5ff4897`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:05:05,989::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:05:05,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3409269-7014-4c5e-8503-d430dd010575`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:05:05,990::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:05:05,991::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e0a98b6-9386-40b2-bfe9-e3c7e4dc34a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:05:05,991::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:05:05,992::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:05:05,992::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e0a98b6-9386-40b2-bfe9-e3c7e4dc34a7`::Granted request >Thread-34::DEBUG::2014-08-11 15:05:05,992::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d3409269-7014-4c5e-8503-d430dd010575`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:05:05,993::task::993::Storage.TaskManager.Task::(_decref) Task=`d3409269-7014-4c5e-8503-d430dd010575`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:05:06,013::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157740032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:06,013::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3409269-7014-4c5e-8503-d430dd010575`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157740032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:06,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3409269-7014-4c5e-8503-d430dd010575`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:05:06,013::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:05:06,014::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:05:06,014::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:05:06,014::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:05:06,014::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:05:06,015::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:05:06,015::task::993::Storage.TaskManager.Task::(_decref) Task=`d3409269-7014-4c5e-8503-d430dd010575`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:05:07,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:07,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132362 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:08,731::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:08,732::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:08,795::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:08,796::__init__::232::IOProcess::(_processLogs) (6295) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:08,796::__init__::232::IOProcess::(_processLogs) (6295) Queuing response >Thread-16::DEBUG::2014-08-11 15:05:08,797::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-110::DEBUG::2014-08-11 15:05:08,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d6539ae-0fd0-4fc9-8cd4-a1df2e7893ba`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:08,818::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-110::INFO::2014-08-11 15:05:08,818::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00163824', 'lastCheck': '10.1', 'valid': True}} >Thread-110::DEBUG::2014-08-11 15:05:08,819::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d6539ae-0fd0-4fc9-8cd4-a1df2e7893ba`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00163824', 'lastCheck': '10.1', 'valid': True}} >Thread-110::DEBUG::2014-08-11 15:05:08,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d6539ae-0fd0-4fc9-8cd4-a1df2e7893ba`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:08,819::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:08,819::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:08,820::task::993::Storage.TaskManager.Task::(_decref) Task=`8d6539ae-0fd0-4fc9-8cd4-a1df2e7893ba`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:08,822::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:05:08,823::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0062077 s, 73.3 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:08,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:08,829::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:08,831::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:08,833::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:08,837::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:08,839::__init__::232::IOProcess::(_processLogs) (6296) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:08,843::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6296) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:08,845::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:08,847::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:08,849::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:08,850::__init__::232::IOProcess::(_processLogs) (6297) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:08,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6297) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:08,855::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:08,855::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:08,856::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:08,856::__init__::232::IOProcess::(_processLogs) (6298) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:08,856::__init__::232::IOProcess::(_processLogs) (6298) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:08,857::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:08,857::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:08,857::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:08,858::__init__::232::IOProcess::(_processLogs) (6299) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:08,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6299) Queuing response >Thread-110::DEBUG::2014-08-11 15:05:08,870::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:09,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:09,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131741 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:11,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:11,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132347 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:13,848::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:13,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:13,850::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:13,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:13,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133326 s, 76.8 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:05:15,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bffba59-f1e2-4dfe-8c10-ca16d272a94d`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:05:15,289::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:05:15,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:15,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:15,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:15,290::__init__::232::IOProcess::(_processLogs) (6300) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:05:15,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6300) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:15,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:15,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 15:05:15,293::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:05:15,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:05:15,295::task::1191::Storage.TaskManager.Task::(prepare) Task=`0bffba59-f1e2-4dfe-8c10-ca16d272a94d`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:05:15,296::__init__::232::IOProcess::(_processLogs) (6301) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:05:15,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bffba59-f1e2-4dfe-8c10-ca16d272a94d`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:05:15,297::__init__::232::IOProcess::(_processLogs) (6301) Queuing response >Thread-55::DEBUG::2014-08-11 15:05:15,297::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:05:15,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:05:15,299::task::993::Storage.TaskManager.Task::(_decref) Task=`0bffba59-f1e2-4dfe-8c10-ca16d272a94d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:05:15,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:15,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132283 s, 77.4 MB/s\n'; <rc> = 0 >Thread-110::DEBUG::2014-08-11 15:05:16,028::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-110::DEBUG::2014-08-11 15:05:16,029::task::595::Storage.TaskManager.Task::(_updateState) Task=`77ec4fa9-efc7-4e41-b1c7-0cc58c155e6f`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:16,029::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:05:16,042::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:16,043::task::1191::Storage.TaskManager.Task::(prepare) Task=`77ec4fa9-efc7-4e41-b1c7-0cc58c155e6f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:16,043::task::595::Storage.TaskManager.Task::(_updateState) Task=`77ec4fa9-efc7-4e41-b1c7-0cc58c155e6f`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:16,043::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:16,044::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:16,044::task::993::Storage.TaskManager.Task::(_decref) Task=`77ec4fa9-efc7-4e41-b1c7-0cc58c155e6f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:05:16,054::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:05:16,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c9f04ec-7738-4129-8a5c-1436d9a7e6ed`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:05:16,055::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:05:16,056::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9ba07b82-88cb-42ee-9ff3-fc7b994bb1f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:05:16,056::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:05:16,056::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:05:16,056::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9ba07b82-88cb-42ee-9ff3-fc7b994bb1f5`::Granted request >Thread-34::DEBUG::2014-08-11 15:05:16,060::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8c9f04ec-7738-4129-8a5c-1436d9a7e6ed`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:05:16,060::task::993::Storage.TaskManager.Task::(_decref) Task=`8c9f04ec-7738-4129-8a5c-1436d9a7e6ed`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:05:16,080::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157608960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:16,081::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c9f04ec-7738-4129-8a5c-1436d9a7e6ed`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157608960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:16,082::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c9f04ec-7738-4129-8a5c-1436d9a7e6ed`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:05:16,083::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:05:16,083::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:05:16,083::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:05:16,084::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:05:16,084::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:05:16,084::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:05:16,084::task::993::Storage.TaskManager.Task::(_decref) Task=`8c9f04ec-7738-4129-8a5c-1436d9a7e6ed`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:05:17,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:18,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132475 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:18,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:18,859::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:18,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:18,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:05:18,900::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:05:18,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:18,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:18,919::__init__::232::IOProcess::(_processLogs) (6302) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:18,921::__init__::232::IOProcess::(_processLogs) (6302) Queuing response >Thread-16::DEBUG::2014-08-11 15:05:18,924::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000744469 s, 611 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:18,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:18,926::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:18,926::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:18,926::__init__::232::IOProcess::(_processLogs) (6303) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:18,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6303) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:18,928::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:18,928::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:18,929::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:18,929::__init__::232::IOProcess::(_processLogs) (6304) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:18,929::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6304) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:18,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:18,931::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:18,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:18,932::__init__::232::IOProcess::(_processLogs) (6305) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:18,932::__init__::232::IOProcess::(_processLogs) (6305) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:18,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:18,933::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:18,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:18,933::__init__::232::IOProcess::(_processLogs) (6306) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:18,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6306) Queuing response >Dummy-33::DEBUG::2014-08-11 15:05:20,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:20,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132873 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:22,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:22,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132537 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:23,930::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:23,932::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-110::DEBUG::2014-08-11 15:05:23,960::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9b54336-7dde-4b43-9d1f-fc9e7e11dc8d`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:23,997::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-110::INFO::2014-08-11 15:05:23,999::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000744469', 'lastCheck': '5.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:24,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-110::DEBUG::2014-08-11 15:05:24,000::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9b54336-7dde-4b43-9d1f-fc9e7e11dc8d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000744469', 'lastCheck': '5.1', 'valid': True}} >Thread-110::DEBUG::2014-08-11 15:05:24,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9b54336-7dde-4b43-9d1f-fc9e7e11dc8d`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:24,003::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:24,004::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:24,004::task::993::Storage.TaskManager.Task::(_decref) Task=`b9b54336-7dde-4b43-9d1f-fc9e7e11dc8d`::ref 0 aborting False >Thread-110::DEBUG::2014-08-11 15:05:24,022::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:24,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:24,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132666 s, 77.2 MB/s\n'; <rc> = 0 >Thread-110::DEBUG::2014-08-11 15:05:26,097::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-110::DEBUG::2014-08-11 15:05:26,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`901d937e-993b-4e08-a043-83abf9b3f03c`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:26,099::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:05:26,111::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:26,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`901d937e-993b-4e08-a043-83abf9b3f03c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:26,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`901d937e-993b-4e08-a043-83abf9b3f03c`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:26,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:26,112::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:26,113::task::993::Storage.TaskManager.Task::(_decref) Task=`901d937e-993b-4e08-a043-83abf9b3f03c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:05:26,125::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:05:26,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`b22a866c-9dee-45fe-b9fb-7306f05fafd7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:05:26,126::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:05:26,127::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c3685351-787d-4b2a-8c12-f7d54d48b45f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:05:26,127::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:05:26,127::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:05:26,128::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c3685351-787d-4b2a-8c12-f7d54d48b45f`::Granted request >Thread-34::DEBUG::2014-08-11 15:05:26,128::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b22a866c-9dee-45fe-b9fb-7306f05fafd7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:05:26,128::task::993::Storage.TaskManager.Task::(_decref) Task=`b22a866c-9dee-45fe-b9fb-7306f05fafd7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:05:26,148::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157608960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:26,149::task::1191::Storage.TaskManager.Task::(prepare) Task=`b22a866c-9dee-45fe-b9fb-7306f05fafd7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858157608960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:26,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`b22a866c-9dee-45fe-b9fb-7306f05fafd7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:05:26,149::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:05:26,149::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:05:26,150::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:05:26,150::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:05:26,150::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:05:26,150::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:05:26,151::task::993::Storage.TaskManager.Task::(_decref) Task=`b22a866c-9dee-45fe-b9fb-7306f05fafd7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:05:26,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:26,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132636 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:28,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:28,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013317 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:28,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:28,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:28,995::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:28,996::__init__::232::IOProcess::(_processLogs) (6307) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:28,996::__init__::232::IOProcess::(_processLogs) (6307) Queuing response >Thread-16::DEBUG::2014-08-11 15:05:28,997::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:29,016::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:29,017::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:29,018::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:05:29,020::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420672 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:29,021::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:29,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:29,022::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:29,022::__init__::232::IOProcess::(_processLogs) (6308) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:29,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6308) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:29,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:29,025::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:29,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:29,026::__init__::232::IOProcess::(_processLogs) (6309) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:29,026::__init__::232::IOProcess::(_processLogs) (6309) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:29,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:29,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:29,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:29,027::__init__::232::IOProcess::(_processLogs) (6310) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:29,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6310) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:29,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:29,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:29,031::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:29,031::__init__::232::IOProcess::(_processLogs) (6311) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:29,031::__init__::232::IOProcess::(_processLogs) (6311) Queuing response >Dummy-33::DEBUG::2014-08-11 15:05:30,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:30,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013205 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:32,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:32,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0284671 s, 36.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:34,025::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:34,026::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:34,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:34,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:34,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132804 s, 77.1 MB/s\n'; <rc> = 0 >Thread-110::DEBUG::2014-08-11 15:05:36,163::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-110::DEBUG::2014-08-11 15:05:36,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad27614b-3fd1-4038-aba9-ce8331984ba4`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:36,164::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:05:36,177::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:36,177::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad27614b-3fd1-4038-aba9-ce8331984ba4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:36,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad27614b-3fd1-4038-aba9-ce8331984ba4`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:36,178::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:36,178::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:36,178::task::993::Storage.TaskManager.Task::(_decref) Task=`ad27614b-3fd1-4038-aba9-ce8331984ba4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:05:36,189::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:05:36,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`239b0964-8570-4ff5-bfb0-66999a1968a8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:05:36,190::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:05:36,191::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`467c4cb4-5a55-47b3-b6cd-e0dcba547717`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:05:36,191::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:05:36,191::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:05:36,191::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`467c4cb4-5a55-47b3-b6cd-e0dcba547717`::Granted request >Thread-34::DEBUG::2014-08-11 15:05:36,192::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`239b0964-8570-4ff5-bfb0-66999a1968a8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:05:36,192::task::993::Storage.TaskManager.Task::(_decref) Task=`239b0964-8570-4ff5-bfb0-66999a1968a8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:05:36,212::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:36,212::task::1191::Storage.TaskManager.Task::(prepare) Task=`239b0964-8570-4ff5-bfb0-66999a1968a8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:36,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`239b0964-8570-4ff5-bfb0-66999a1968a8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:05:36,213::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:05:36,213::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:05:36,213::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:05:36,214::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:05:36,214::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:05:36,214::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:05:36,214::task::993::Storage.TaskManager.Task::(_decref) Task=`239b0964-8570-4ff5-bfb0-66999a1968a8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:05:36,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:36,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01329 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:38,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:38,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131754 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:39,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:39,032::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:39,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:05:39,097::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:05:39,097::__init__::232::IOProcess::(_processLogs) (6312) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:39,117::__init__::232::IOProcess::(_processLogs) (6312) Queuing response >Thread-16::DEBUG::2014-08-11 15:05:39,121::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00078078 s, 583 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:39,122::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:39,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:39,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:39,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:39,130::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:39,131::__init__::232::IOProcess::(_processLogs) (6313) Got request for method 'statvfs' >Thread-110::DEBUG::2014-08-11 15:05:39,131::task::595::Storage.TaskManager.Task::(_updateState) Task=`bebc9e31-120e-47c1-9a2d-771a152e37c7`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:05:39,134::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6313) Queuing response >Thread-110::INFO::2014-08-11 15:05:39,135::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:39,138::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-110::INFO::2014-08-11 15:05:39,138::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420672', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 15:05:39,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-110::DEBUG::2014-08-11 15:05:39,140::task::1191::Storage.TaskManager.Task::(prepare) Task=`bebc9e31-120e-47c1-9a2d-771a152e37c7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420672', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 15:05:39,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110::DEBUG::2014-08-11 15:05:39,142::task::595::Storage.TaskManager.Task::(_updateState) Task=`bebc9e31-120e-47c1-9a2d-771a152e37c7`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:05:39,142::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-110::DEBUG::2014-08-11 15:05:39,143::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:05:39,144::__init__::232::IOProcess::(_processLogs) (6314) Got request for method 'access' >Thread-110::DEBUG::2014-08-11 15:05:39,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 15:05:39,147::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6314) Queuing response >Thread-110::DEBUG::2014-08-11 15:05:39,147::task::993::Storage.TaskManager.Task::(_decref) Task=`bebc9e31-120e-47c1-9a2d-771a152e37c7`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:05:39,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:39,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:39,154::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:39,156::__init__::232::IOProcess::(_processLogs) (6315) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:39,157::__init__::232::IOProcess::(_processLogs) (6315) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:39,160::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:39,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:39,164::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:39,165::__init__::232::IOProcess::(_processLogs) (6316) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:39,167::__init__::232::IOProcess::(_processLogs) (6316) Queuing response >Thread-110::DEBUG::2014-08-11 15:05:39,188::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:40,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:40,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132827 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:42,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:42,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131805 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:44,153::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:44,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:44,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:44,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:44,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132473 s, 77.3 MB/s\n'; <rc> = 0 >Thread-110::DEBUG::2014-08-11 15:05:46,227::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-110::DEBUG::2014-08-11 15:05:46,228::task::595::Storage.TaskManager.Task::(_updateState) Task=`9aea66ff-465e-4e98-b2c5-e1f3001398b7`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:46,229::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:05:46,241::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:46,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`9aea66ff-465e-4e98-b2c5-e1f3001398b7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:46,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`9aea66ff-465e-4e98-b2c5-e1f3001398b7`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:46,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:46,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:46,243::task::993::Storage.TaskManager.Task::(_decref) Task=`9aea66ff-465e-4e98-b2c5-e1f3001398b7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:05:46,253::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:05:46,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`76907919-2081-4b3d-aafa-2d844117897e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:05:46,254::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:05:46,255::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eb396cd6-b5f2-4412-a81f-948aba400000`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:05:46,255::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:05:46,255::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:05:46,255::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eb396cd6-b5f2-4412-a81f-948aba400000`::Granted request >Thread-34::DEBUG::2014-08-11 15:05:46,256::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`76907919-2081-4b3d-aafa-2d844117897e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:05:46,256::task::993::Storage.TaskManager.Task::(_decref) Task=`76907919-2081-4b3d-aafa-2d844117897e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:05:46,276::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:46,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`76907919-2081-4b3d-aafa-2d844117897e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:46,277::task::595::Storage.TaskManager.Task::(_updateState) Task=`76907919-2081-4b3d-aafa-2d844117897e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:05:46,277::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:05:46,277::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:05:46,277::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:05:46,278::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:05:46,278::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:05:46,278::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:05:46,278::task::993::Storage.TaskManager.Task::(_decref) Task=`76907919-2081-4b3d-aafa-2d844117897e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:05:46,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:47,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131875 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:49,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:49,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133261 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:49,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:49,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:49,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:49,197::__init__::232::IOProcess::(_processLogs) (6317) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:05:49,197::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:05:49,199::__init__::232::IOProcess::(_processLogs) (6317) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:49,218::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:49,219::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:49,220::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:05:49,222::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000479671 s, 949 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:49,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:49,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:49,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:49,224::__init__::232::IOProcess::(_processLogs) (6318) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:49,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6318) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:49,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:49,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:49,227::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:49,228::__init__::232::IOProcess::(_processLogs) (6319) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:49,228::__init__::232::IOProcess::(_processLogs) (6319) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:49,228::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:49,229::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:49,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:49,230::__init__::232::IOProcess::(_processLogs) (6320) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:49,230::__init__::232::IOProcess::(_processLogs) (6320) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:49,230::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:49,231::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:49,232::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:49,232::__init__::232::IOProcess::(_processLogs) (6321) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:49,233::__init__::232::IOProcess::(_processLogs) (6321) Queuing response >Dummy-33::DEBUG::2014-08-11 15:05:51,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:51,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218559 s, 46.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:05:53,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:53,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131649 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:54,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:54,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:54,226::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-110::DEBUG::2014-08-11 15:05:54,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`6be4e6e1-4b69-4578-81e0-215a9ace464c`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:54,275::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-110::INFO::2014-08-11 15:05:54,276::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000479671', 'lastCheck': '5.0', 'valid': True}} >Thread-110::DEBUG::2014-08-11 15:05:54,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`6be4e6e1-4b69-4578-81e0-215a9ace464c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000479671', 'lastCheck': '5.0', 'valid': True}} >Thread-110::DEBUG::2014-08-11 15:05:54,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`6be4e6e1-4b69-4578-81e0-215a9ace464c`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:54,277::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:54,277::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:54,277::task::993::Storage.TaskManager.Task::(_decref) Task=`6be4e6e1-4b69-4578-81e0-215a9ace464c`::ref 0 aborting False >Thread-110::DEBUG::2014-08-11 15:05:54,297::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:05:55,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:55,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131377 s, 77.9 MB/s\n'; <rc> = 0 >Thread-110::DEBUG::2014-08-11 15:05:56,291::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-110::DEBUG::2014-08-11 15:05:56,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a72620d-e5b6-4991-8fc3-8740381629a2`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:05:56,292::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:05:56,305::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:56,305::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a72620d-e5b6-4991-8fc3-8740381629a2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:05:56,306::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a72620d-e5b6-4991-8fc3-8740381629a2`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:05:56,306::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:05:56,306::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:05:56,306::task::993::Storage.TaskManager.Task::(_decref) Task=`1a72620d-e5b6-4991-8fc3-8740381629a2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:05:56,316::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:05:56,317::task::595::Storage.TaskManager.Task::(_updateState) Task=`493b2c73-96bd-472c-914b-895f8798301a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:05:56,318::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:05:56,318::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`49e41a4d-ed7f-4257-b048-8c3cf928fc7a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:05:56,319::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:05:56,319::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:05:56,319::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`49e41a4d-ed7f-4257-b048-8c3cf928fc7a`::Granted request >Thread-34::DEBUG::2014-08-11 15:05:56,320::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`493b2c73-96bd-472c-914b-895f8798301a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:05:56,320::task::993::Storage.TaskManager.Task::(_decref) Task=`493b2c73-96bd-472c-914b-895f8798301a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:05:56,340::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:56,340::task::1191::Storage.TaskManager.Task::(prepare) Task=`493b2c73-96bd-472c-914b-895f8798301a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:05:56,341::task::595::Storage.TaskManager.Task::(_updateState) Task=`493b2c73-96bd-472c-914b-895f8798301a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:05:56,341::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:05:56,341::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:05:56,341::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:05:56,342::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:05:56,342::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:05:56,342::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:05:56,342::task::993::Storage.TaskManager.Task::(_decref) Task=`493b2c73-96bd-472c-914b-895f8798301a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:05:57,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:05:57,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131777 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:59,230::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:59,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:59,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:59,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:59,300::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:05:59,300::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:05:59,320::__init__::232::IOProcess::(_processLogs) (6322) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:05:59,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:05:59,322::__init__::232::IOProcess::(_processLogs) (6322) Queuing response >Dummy-33::DEBUG::2014-08-11 15:05:59,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 15:05:59,325::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00140514 s, 324 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:59,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:59,347::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:59,347::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:59,348::__init__::232::IOProcess::(_processLogs) (6323) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:05:59,355::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6323) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:59,358::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:59,360::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:59,361::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:59,361::__init__::232::IOProcess::(_processLogs) (6324) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:59,361::__init__::232::IOProcess::(_processLogs) (6324) Queuing response >Thread-14::DEBUG::2014-08-11 15:05:59,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:59,363::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:59,367::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:59,368::__init__::232::IOProcess::(_processLogs) (6325) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:59,370::__init__::232::IOProcess::(_processLogs) (6325) Queuing response >Dummy-33::DEBUG::2014-08-11 15:05:59,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265743 s, 38.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:05:59,373::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:05:59,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:05:59,376::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:05:59,376::__init__::232::IOProcess::(_processLogs) (6326) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:05:59,377::__init__::232::IOProcess::(_processLogs) (6326) Queuing response >Dummy-33::DEBUG::2014-08-11 15:06:01,401::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:01,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134696 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:03,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:03,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133772 s, 76.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:04,351::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:04,352::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:04,397::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:05,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:05,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135209 s, 75.7 MB/s\n'; <rc> = 0 >Thread-110::DEBUG::2014-08-11 15:06:06,361::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-110::DEBUG::2014-08-11 15:06:06,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7da833a-3976-4112-9878-36472b35a28d`::moving from state init -> state preparing >Thread-110::INFO::2014-08-11 15:06:06,363::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-110::INFO::2014-08-11 15:06:06,376::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:06:06,377::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7da833a-3976-4112-9878-36472b35a28d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-110::DEBUG::2014-08-11 15:06:06,377::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7da833a-3976-4112-9878-36472b35a28d`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-08-11 15:06:06,377::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-08-11 15:06:06,377::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-08-11 15:06:06,378::task::993::Storage.TaskManager.Task::(_decref) Task=`d7da833a-3976-4112-9878-36472b35a28d`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 15:06:06,402::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:35832 >Detector thread::DEBUG::2014-08-11 15:06:06,403::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:35832 >Detector thread::DEBUG::2014-08-11 15:06:06,403::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:35832 >Detector thread::DEBUG::2014-08-11 15:06:06,403::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 35832) >Thread-34::DEBUG::2014-08-11 15:06:06,409::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:06:06,410::task::595::Storage.TaskManager.Task::(_updateState) Task=`21d10c97-7ca1-4b6b-948f-d3fda52bafc9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:06:06,410::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:06:06,411::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d329ead0-ae14-4b0d-80a1-0db7197dc101`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:06:06,411::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:06:06,412::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:06:06,412::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d329ead0-ae14-4b0d-80a1-0db7197dc101`::Granted request >Thread-34::DEBUG::2014-08-11 15:06:06,412::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`21d10c97-7ca1-4b6b-948f-d3fda52bafc9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:06:06,413::task::993::Storage.TaskManager.Task::(_decref) Task=`21d10c97-7ca1-4b6b-948f-d3fda52bafc9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:06:06,432::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:06,433::task::1191::Storage.TaskManager.Task::(prepare) Task=`21d10c97-7ca1-4b6b-948f-d3fda52bafc9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:06,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`21d10c97-7ca1-4b6b-948f-d3fda52bafc9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:06:06,433::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:06:06,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:06:06,434::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:06:06,434::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:06:06,434::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:06:06,434::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:06:06,435::task::993::Storage.TaskManager.Task::(_decref) Task=`21d10c97-7ca1-4b6b-948f-d3fda52bafc9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:06:07,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:07,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013326 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:09,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:09,377::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:09,398::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:06:09,399::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:06:09,401::__init__::232::IOProcess::(_processLogs) (6327) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:09,418::__init__::232::IOProcess::(_processLogs) (6327) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:09,421::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:09,422::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:09,426::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:06:09,429::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00055167 s, 825 kB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:06:09,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec25fe33-7c38-4207-84f4-94c7b0bd5e9c`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:09,433::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-14::DEBUG::2014-08-11 15:06:09,434::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-111::INFO::2014-08-11 15:06:09,435::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00140514', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 15:06:09,435::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-111::DEBUG::2014-08-11 15:06:09,437::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec25fe33-7c38-4207-84f4-94c7b0bd5e9c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00140514', 'lastCheck': '10.1', 'valid': True}} >Thread-14::DEBUG::2014-08-11 15:06:09,437::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-111::DEBUG::2014-08-11 15:06:09,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec25fe33-7c38-4207-84f4-94c7b0bd5e9c`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:06:09,438::__init__::232::IOProcess::(_processLogs) (6328) Got request for method 'statvfs' >Thread-111::DEBUG::2014-08-11 15:06:09,439::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:06:09,440::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6328) Queuing response >Thread-111::DEBUG::2014-08-11 15:06:09,440::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:09,443::task::993::Storage.TaskManager.Task::(_decref) Task=`ec25fe33-7c38-4207-84f4-94c7b0bd5e9c`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:06:09,443::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:09,445::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:09,446::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:09,448::__init__::232::IOProcess::(_processLogs) (6329) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:09,450::__init__::232::IOProcess::(_processLogs) (6329) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:09,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:09,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:09,456::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:09,457::__init__::232::IOProcess::(_processLogs) (6330) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:09,458::__init__::232::IOProcess::(_processLogs) (6330) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:09,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:09,463::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:09,465::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:09,466::__init__::232::IOProcess::(_processLogs) (6331) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:09,468::__init__::232::IOProcess::(_processLogs) (6331) Queuing response >Thread-111::DEBUG::2014-08-11 15:06:09,484::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:09,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:09,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135184 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:11,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:11,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132431 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:13,733::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:13,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134855 s, 75.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:14,437::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:14,439::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:14,498::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 15:06:15,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`af379aa4-d356-4c47-b43e-6457571c4e0e`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:06:15,424::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:06:15,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:15,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:15,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:15,426::__init__::232::IOProcess::(_processLogs) (6332) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:06:15,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6332) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:15,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:15,429::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 15:06:15,429::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:06:15,430::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:06:15,431::task::1191::Storage.TaskManager.Task::(prepare) Task=`af379aa4-d356-4c47-b43e-6457571c4e0e`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:06:15,431::__init__::232::IOProcess::(_processLogs) (6333) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:06:15,432::task::595::Storage.TaskManager.Task::(_updateState) Task=`af379aa4-d356-4c47-b43e-6457571c4e0e`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:06:15,433::__init__::232::IOProcess::(_processLogs) (6333) Queuing response >Thread-55::DEBUG::2014-08-11 15:06:15,433::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:06:15,434::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:06:15,434::task::993::Storage.TaskManager.Task::(_decref) Task=`af379aa4-d356-4c47-b43e-6457571c4e0e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:06:15,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:15,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132641 s, 77.2 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:06:16,464::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:06:16,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`b59603b9-dce7-4dd1-b078-81257318a786`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:16,467::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:06:16,495::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:16,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`b59603b9-dce7-4dd1-b078-81257318a786`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:16,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`b59603b9-dce7-4dd1-b078-81257318a786`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:16,499::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:16,499::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:16,499::task::993::Storage.TaskManager.Task::(_decref) Task=`b59603b9-dce7-4dd1-b078-81257318a786`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:06:16,511::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:06:16,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`81fd3d0d-04ec-4daa-b62e-107cc5de3e52`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:06:16,512::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:06:16,513::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8ee4762-e0d1-4e0c-bbbe-48b2fd29bbaa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:06:16,513::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:06:16,513::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:06:16,514::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8ee4762-e0d1-4e0c-bbbe-48b2fd29bbaa`::Granted request >Thread-34::DEBUG::2014-08-11 15:06:16,514::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`81fd3d0d-04ec-4daa-b62e-107cc5de3e52`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:06:16,514::task::993::Storage.TaskManager.Task::(_decref) Task=`81fd3d0d-04ec-4daa-b62e-107cc5de3e52`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:06:16,536::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:16,536::task::1191::Storage.TaskManager.Task::(prepare) Task=`81fd3d0d-04ec-4daa-b62e-107cc5de3e52`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136506368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:16,537::task::595::Storage.TaskManager.Task::(_updateState) Task=`81fd3d0d-04ec-4daa-b62e-107cc5de3e52`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:06:16,537::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:06:16,537::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:06:16,537::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:06:16,538::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:06:16,538::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:06:16,538::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:06:16,539::task::993::Storage.TaskManager.Task::(_decref) Task=`81fd3d0d-04ec-4daa-b62e-107cc5de3e52`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:06:17,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:17,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132088 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:19,464::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:19,464::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:19,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:06:19,499::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:06:19,499::__init__::232::IOProcess::(_processLogs) (6334) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:19,517::__init__::232::IOProcess::(_processLogs) (6334) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:19,521::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:06:19,521::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000764212 s, 595 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:19,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:19,526::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:06:19,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:19,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:19,530::__init__::232::IOProcess::(_processLogs) (6335) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:19,531::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:06:19,533::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6335) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:19,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:19,536::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:19,536::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:19,536::__init__::232::IOProcess::(_processLogs) (6336) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:19,537::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6336) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:19,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:19,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:19,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:19,538::__init__::232::IOProcess::(_processLogs) (6337) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:19,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6337) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:19,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:19,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:19,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:19,541::__init__::232::IOProcess::(_processLogs) (6338) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:19,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6338) Queuing response >Dummy-33::DEBUG::2014-08-11 15:06:19,897::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:19,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133567 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:21,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:21,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131899 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:24,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:24,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132449 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:24,541::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:24,543::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:06:24,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`d65239c4-1ff2-46ec-ae13-c7d62a4eb9ac`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:24,599::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:06:24,601::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000764212', 'lastCheck': '5.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:24,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:06:24,603::task::1191::Storage.TaskManager.Task::(prepare) Task=`d65239c4-1ff2-46ec-ae13-c7d62a4eb9ac`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000764212', 'lastCheck': '5.1', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:06:24,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`d65239c4-1ff2-46ec-ae13-c7d62a4eb9ac`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:24,605::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:24,606::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:24,607::task::993::Storage.TaskManager.Task::(_decref) Task=`d65239c4-1ff2-46ec-ae13-c7d62a4eb9ac`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:06:24,626::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:26,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:26,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181482 s, 56.4 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:06:26,551::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:06:26,552::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e390399-e47b-4f5c-8fa3-ceecd7e890a0`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:26,552::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:06:26,565::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:26,566::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e390399-e47b-4f5c-8fa3-ceecd7e890a0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:26,566::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e390399-e47b-4f5c-8fa3-ceecd7e890a0`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:26,566::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:26,566::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:26,567::task::993::Storage.TaskManager.Task::(_decref) Task=`6e390399-e47b-4f5c-8fa3-ceecd7e890a0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:06:26,577::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:06:26,578::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a8f0766-2157-42fe-82c9-2ce7474c09cb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:06:26,578::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:06:26,579::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`12aeb7f1-da0f-4a43-83dc-14ad4789aa14`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:06:26,579::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:06:26,580::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:06:26,580::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`12aeb7f1-da0f-4a43-83dc-14ad4789aa14`::Granted request >Thread-34::DEBUG::2014-08-11 15:06:26,580::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5a8f0766-2157-42fe-82c9-2ce7474c09cb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:06:26,581::task::993::Storage.TaskManager.Task::(_decref) Task=`5a8f0766-2157-42fe-82c9-2ce7474c09cb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:06:26,609::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:26,611::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a8f0766-2157-42fe-82c9-2ce7474c09cb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:26,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a8f0766-2157-42fe-82c9-2ce7474c09cb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:06:26,612::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:06:26,612::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:06:26,613::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:06:26,614::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:06:26,614::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:06:26,615::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:06:26,615::task::993::Storage.TaskManager.Task::(_decref) Task=`5a8f0766-2157-42fe-82c9-2ce7474c09cb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:06:28,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:28,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132605 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:29,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:29,544::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:29,544::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:29,544::__init__::232::IOProcess::(_processLogs) (6339) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:29,544::__init__::232::IOProcess::(_processLogs) (6339) Queuing response >Thread-16::DEBUG::2014-08-11 15:06:29,545::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:06:29,564::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425034 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:29,565::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:29,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:29,566::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:29,566::__init__::232::IOProcess::(_processLogs) (6340) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:29,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6340) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:29,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:29,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:29,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:29,568::__init__::232::IOProcess::(_processLogs) (6341) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:29,569::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6341) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:29,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:29,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:29,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:29,571::__init__::232::IOProcess::(_processLogs) (6342) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:29,571::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|(6342) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:29,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:29,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:29,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:29,573::__init__::232::IOProcess::(_processLogs) (6343) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:29,573::__init__::232::IOProcess::(_processLogs) (6343) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:29,612::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:29,614::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:29,615::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:30,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:30,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133362 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:32,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:32,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0216607 s, 47.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:34,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:34,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130874 s, 78.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:34,621::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:34,622::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:34,699::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:36,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:36,510::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133314 s, 76.8 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:06:36,627::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:06:36,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d7a4973-ccc0-4ca9-99c9-2fd691df447c`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:36,629::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:06:36,641::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:36,642::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d7a4973-ccc0-4ca9-99c9-2fd691df447c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:36,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d7a4973-ccc0-4ca9-99c9-2fd691df447c`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:36,642::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:36,642::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:36,643::task::993::Storage.TaskManager.Task::(_decref) Task=`9d7a4973-ccc0-4ca9-99c9-2fd691df447c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:06:36,653::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:06:36,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`23e8d6a6-a6c4-432c-8ac4-61b33134c3b4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:06:36,654::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:06:36,655::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ab686c53-1039-4514-82ee-6a9e91440e38`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:06:36,655::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:06:36,655::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:06:36,656::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ab686c53-1039-4514-82ee-6a9e91440e38`::Granted request >Thread-34::DEBUG::2014-08-11 15:06:36,656::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`23e8d6a6-a6c4-432c-8ac4-61b33134c3b4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:06:36,656::task::993::Storage.TaskManager.Task::(_decref) Task=`23e8d6a6-a6c4-432c-8ac4-61b33134c3b4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:06:36,676::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:36,677::task::1191::Storage.TaskManager.Task::(prepare) Task=`23e8d6a6-a6c4-432c-8ac4-61b33134c3b4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:36,677::task::595::Storage.TaskManager.Task::(_updateState) Task=`23e8d6a6-a6c4-432c-8ac4-61b33134c3b4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:06:36,677::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:06:36,677::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:06:36,678::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:06:36,678::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:06:36,678::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:06:36,678::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:06:36,679::task::993::Storage.TaskManager.Task::(_decref) Task=`23e8d6a6-a6c4-432c-8ac4-61b33134c3b4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:06:38,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:38,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134711 s, 76.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:39,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:39,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:39,576::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:39,576::__init__::232::IOProcess::(_processLogs) (6344) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:06:39,577::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:06:39,596::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042119 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:39,597::__init__::232::IOProcess::(_processLogs) (6345) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:39,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:39,599::__init__::232::IOProcess::(_processLogs) (6346) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:39,600::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:39,600::__init__::232::IOProcess::(_processLogs) (6347) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:39,601::__init__::232::IOProcess::(_processLogs) (6347) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:39,601::__init__::232::IOProcess::(_processLogs) (6347) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:39,602::__init__::232::IOProcess::(_processLogs) (6347) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:39,603::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:39,603::__init__::232::IOProcess::(_processLogs) (6348) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:39,604::__init__::232::IOProcess::(_processLogs) (6348) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:39,705::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:39,707::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:06:39,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`8eef981b-a26a-4f69-a923-a09b04def49c`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:39,800::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:06:39,802::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042119', 'lastCheck': '0.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:39,803::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:06:39,803::task::1191::Storage.TaskManager.Task::(prepare) Task=`8eef981b-a26a-4f69-a923-a09b04def49c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042119', 'lastCheck': '0.2', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:06:39,804::task::595::Storage.TaskManager.Task::(_updateState) Task=`8eef981b-a26a-4f69-a923-a09b04def49c`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:39,806::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:39,807::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:39,807::task::993::Storage.TaskManager.Task::(_decref) Task=`8eef981b-a26a-4f69-a923-a09b04def49c`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:06:39,827::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:40,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:40,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132565 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:42,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:42,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131446 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:44,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:44,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0143236 s, 71.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:44,814::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:44,815::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:44,816::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:06:46,692::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:06:46,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`14f0441b-1cac-4edb-83d6-c08b88d3b637`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:46,693::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:06:46,706::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:46,706::task::1191::Storage.TaskManager.Task::(prepare) Task=`14f0441b-1cac-4edb-83d6-c08b88d3b637`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:46,707::task::595::Storage.TaskManager.Task::(_updateState) Task=`14f0441b-1cac-4edb-83d6-c08b88d3b637`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:46,707::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:46,707::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:46,707::task::993::Storage.TaskManager.Task::(_decref) Task=`14f0441b-1cac-4edb-83d6-c08b88d3b637`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:06:46,718::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:06:46,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b3c486d-0aab-4413-81d4-842f880b3377`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:06:46,719::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:06:46,720::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c0fae011-7033-4313-b67a-64cc45402175`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:06:46,720::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:06:46,721::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:06:46,721::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c0fae011-7033-4313-b67a-64cc45402175`::Granted request >Thread-34::DEBUG::2014-08-11 15:06:46,722::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6b3c486d-0aab-4413-81d4-842f880b3377`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:06:46,722::task::993::Storage.TaskManager.Task::(_decref) Task=`6b3c486d-0aab-4413-81d4-842f880b3377`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:06:46,743::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:46,743::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b3c486d-0aab-4413-81d4-842f880b3377`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:46,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b3c486d-0aab-4413-81d4-842f880b3377`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:06:46,744::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:06:46,744::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:06:46,744::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:06:46,745::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:06:46,745::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:06:46,745::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:06:46,745::task::993::Storage.TaskManager.Task::(_decref) Task=`6b3c486d-0aab-4413-81d4-842f880b3377`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:06:46,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:46,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131383 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:48,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:48,867::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135289 s, 75.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:49,604::__init__::232::IOProcess::(_processLogs) (6347) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:49,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:49,605::__init__::232::IOProcess::(_processLogs) (6347) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:06:49,606::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:06:49,624::__init__::232::IOProcess::(_processLogs) (6349) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:49,626::__init__::232::IOProcess::(_processLogs) (6349) Queuing response >Thread-16::DEBUG::2014-08-11 15:06:49,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000616389 s, 738 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:49,628::__init__::232::IOProcess::(_processLogs) (6347) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:49,628::__init__::232::IOProcess::(_processLogs) (6347) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:49,628::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:49,629::__init__::232::IOProcess::(_processLogs) (6350) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:49,630::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:49,630::__init__::232::IOProcess::(_processLogs) (6351) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:49,631::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:49,631::__init__::232::IOProcess::(_processLogs) (6352) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:49,632::__init__::232::IOProcess::(_processLogs) (6352) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:49,633::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:49,633::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:49,634::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:49,635::__init__::232::IOProcess::(_processLogs) (6353) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:49,635::__init__::232::IOProcess::(_processLogs) (6353) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:49,824::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:49,825::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:49,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:50,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:50,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133142 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:52,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:52,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132357 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:54,830::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:54,831::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:54,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:06:54,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c4fadfd-3e77-45f9-ab01-53668fa6dcae`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:54,927::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:06:54,927::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000616389', 'lastCheck': '5.3', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:06:54,928::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c4fadfd-3e77-45f9-ab01-53668fa6dcae`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000616389', 'lastCheck': '5.3', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:06:54,928::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c4fadfd-3e77-45f9-ab01-53668fa6dcae`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:54,928::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:54,928::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:54,928::task::993::Storage.TaskManager.Task::(_decref) Task=`4c4fadfd-3e77-45f9-ab01-53668fa6dcae`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:06:54,947::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:06:55,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:55,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133127 s, 76.9 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:06:56,757::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:06:56,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`242edab9-0952-4a14-8c68-b106d6d2a7a5`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:06:56,759::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:06:56,771::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:56,772::task::1191::Storage.TaskManager.Task::(prepare) Task=`242edab9-0952-4a14-8c68-b106d6d2a7a5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:06:56,772::task::595::Storage.TaskManager.Task::(_updateState) Task=`242edab9-0952-4a14-8c68-b106d6d2a7a5`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:06:56,772::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:06:56,772::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:06:56,773::task::993::Storage.TaskManager.Task::(_decref) Task=`242edab9-0952-4a14-8c68-b106d6d2a7a5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:06:56,782::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:06:56,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`555c7a32-d07e-4781-a347-844e96608bef`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:06:56,783::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:06:56,784::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`de37e4ef-0738-4ae0-bced-042cc7b22010`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:06:56,784::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:06:56,785::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:06:56,785::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`de37e4ef-0738-4ae0-bced-042cc7b22010`::Granted request >Thread-34::DEBUG::2014-08-11 15:06:56,786::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`555c7a32-d07e-4781-a347-844e96608bef`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:06:56,786::task::993::Storage.TaskManager.Task::(_decref) Task=`555c7a32-d07e-4781-a347-844e96608bef`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:06:56,806::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:56,806::task::1191::Storage.TaskManager.Task::(prepare) Task=`555c7a32-d07e-4781-a347-844e96608bef`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:06:56,806::task::595::Storage.TaskManager.Task::(_updateState) Task=`555c7a32-d07e-4781-a347-844e96608bef`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:06:56,806::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:06:56,807::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:06:56,807::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:06:56,807::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:06:56,808::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:06:56,808::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:06:56,808::task::993::Storage.TaskManager.Task::(_decref) Task=`555c7a32-d07e-4781-a347-844e96608bef`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:06:57,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:57,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0142047 s, 72.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:06:59,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:06:59,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133584 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:59,635::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:59,636::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:06:59,638::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:06:59,638::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:59,655::__init__::232::IOProcess::(_processLogs) (6354) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:59,656::__init__::232::IOProcess::(_processLogs) (6354) Queuing response >Thread-16::DEBUG::2014-08-11 15:06:59,657::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000906136 s, 502 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:06:59,658::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:59,658::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:59,659::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:59,659::__init__::232::IOProcess::(_processLogs) (6355) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:06:59,660::__init__::232::IOProcess::(_processLogs) DEBUG|(6355) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:59,661::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:59,661::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:59,662::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:59,662::__init__::232::IOProcess::(_processLogs) (6356) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:59,662::__init__::232::IOProcess::(_processLogs) DEBUG|(6356) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:59,664::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:59,664::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:59,664::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:59,664::__init__::232::IOProcess::(_processLogs) (6357) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:59,665::__init__::232::IOProcess::(_processLogs) DEBUG|(6357) Queuing response >Thread-14::DEBUG::2014-08-11 15:06:59,666::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:06:59,666::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:06:59,666::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:06:59,667::__init__::232::IOProcess::(_processLogs) (6358) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:06:59,667::__init__::232::IOProcess::(_processLogs) DEBUG|(6358) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:59,838::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:59,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:06:59,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:01,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:01,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133143 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:03,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:03,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132774 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:04,844::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:04,846::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:04,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:05,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:05,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132071 s, 77.5 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:07:06,819::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:07:06,820::task::595::Storage.TaskManager.Task::(_updateState) Task=`84392ff9-d89e-4c84-a81e-81025151886f`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:06,820::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:07:06,833::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:06,834::task::1191::Storage.TaskManager.Task::(prepare) Task=`84392ff9-d89e-4c84-a81e-81025151886f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:06,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`84392ff9-d89e-4c84-a81e-81025151886f`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:06,834::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:06,834::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:06,835::task::993::Storage.TaskManager.Task::(_decref) Task=`84392ff9-d89e-4c84-a81e-81025151886f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:07:06,845::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:07:06,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c145588-a5c6-44e2-be95-a9ca6b741d3b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:07:06,846::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:07:06,846::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`559e1e5b-d609-4ec5-9c3d-62a19819ca4e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:07:06,847::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:07:06,847::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:07:06,847::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`559e1e5b-d609-4ec5-9c3d-62a19819ca4e`::Granted request >Thread-34::DEBUG::2014-08-11 15:07:06,848::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9c145588-a5c6-44e2-be95-a9ca6b741d3b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:07:06,848::task::993::Storage.TaskManager.Task::(_decref) Task=`9c145588-a5c6-44e2-be95-a9ca6b741d3b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:07:06,868::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:06,869::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c145588-a5c6-44e2-be95-a9ca6b741d3b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:06,869::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c145588-a5c6-44e2-be95-a9ca6b741d3b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:07:06,869::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:07:06,869::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:07:06,870::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:07:06,870::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:07:06,870::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:07:06,870::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:07:06,871::task::993::Storage.TaskManager.Task::(_decref) Task=`9c145588-a5c6-44e2-be95-a9ca6b741d3b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:07:07,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:07,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132631 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:09,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:09,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013177 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:09,669::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:09,670::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:09,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:09,670::__init__::232::IOProcess::(_processLogs) (6359) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:09,671::__init__::232::IOProcess::(_processLogs) DEBUG|(6359) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:09,672::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:07:09,693::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000960953 s, 473 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:09,694::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:09,695::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:09,695::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:09,695::__init__::232::IOProcess::(_processLogs) (6360) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:09,696::__init__::232::IOProcess::(_processLogs) DEBUG|(6360) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:09,697::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:09,698::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:09,698::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:09,698::__init__::232::IOProcess::(_processLogs) (6361) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:09,699::__init__::232::IOProcess::(_processLogs) DEBUG|(6361) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:09,700::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:09,700::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:09,700::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:09,700::__init__::232::IOProcess::(_processLogs) (6362) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:09,701::__init__::232::IOProcess::(_processLogs) DEBUG|(6362) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:09,702::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:09,702::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:09,702::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:09,703::__init__::232::IOProcess::(_processLogs) (6363) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:09,703::__init__::232::IOProcess::(_processLogs) DEBUG|(6363) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:09,853::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:09,854::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:09,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:07:10,044::task::595::Storage.TaskManager.Task::(_updateState) Task=`6153cb69-4eb0-4223-bae7-bfb6dcb6a34f`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:10,044::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:07:10,045::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000960953', 'lastCheck': '0.3', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:07:10,045::task::1191::Storage.TaskManager.Task::(prepare) Task=`6153cb69-4eb0-4223-bae7-bfb6dcb6a34f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000960953', 'lastCheck': '0.3', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:07:10,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`6153cb69-4eb0-4223-bae7-bfb6dcb6a34f`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:10,045::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:10,045::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:10,046::task::993::Storage.TaskManager.Task::(_decref) Task=`6153cb69-4eb0-4223-bae7-bfb6dcb6a34f`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:07:10,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:11,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:11,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133894 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:13,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:13,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132174 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:14,862::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:14,863::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:14,864::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:15,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:15,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134016 s, 76.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:07:15,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`eefb2c60-13f6-423a-94a9-0e00b516aa07`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:07:15,647::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:07:15,648::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:15,649::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:15,649::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:15,649::__init__::232::IOProcess::(_processLogs) (6364) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:07:15,650::__init__::232::IOProcess::(_processLogs) DEBUG|(6364) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:15,651::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-55::INFO::2014-08-11 15:07:15,652::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:07:15,652::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::DEBUG::2014-08-11 15:07:15,653::task::1191::Storage.TaskManager.Task::(prepare) Task=`eefb2c60-13f6-423a-94a9-0e00b516aa07`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:07:15,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:07:15,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`eefb2c60-13f6-423a-94a9-0e00b516aa07`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:07:15,655::__init__::232::IOProcess::(_processLogs) (6365) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:07:15,655::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:07:15,656::__init__::232::IOProcess::(_processLogs) (6365) Queuing response >Thread-55::DEBUG::2014-08-11 15:07:15,657::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:07:15,658::task::993::Storage.TaskManager.Task::(_decref) Task=`eefb2c60-13f6-423a-94a9-0e00b516aa07`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:07:16,883::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-111::DEBUG::2014-08-11 15:07:16,884::task::595::Storage.TaskManager.Task::(_updateState) Task=`b605dd06-3155-42ff-bace-6b0f7f0cedc3`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:16,884::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:07:16,898::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:16,898::task::1191::Storage.TaskManager.Task::(prepare) Task=`b605dd06-3155-42ff-bace-6b0f7f0cedc3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:16,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`b605dd06-3155-42ff-bace-6b0f7f0cedc3`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:16,898::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:16,899::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:16,899::task::993::Storage.TaskManager.Task::(_decref) Task=`b605dd06-3155-42ff-bace-6b0f7f0cedc3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:07:16,910::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 15:07:16,910::task::595::Storage.TaskManager.Task::(_updateState) Task=`f88a5e16-a561-4c43-8be3-e0be25b90c4a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:07:16,910::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:07:16,911::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a8267bb2-a1d8-4e6d-843f-99e71c8d3241`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:07:16,911::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:07:16,912::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:07:16,912::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a8267bb2-a1d8-4e6d-843f-99e71c8d3241`::Granted request >Thread-34::DEBUG::2014-08-11 15:07:16,914::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f88a5e16-a561-4c43-8be3-e0be25b90c4a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:07:16,914::task::993::Storage.TaskManager.Task::(_decref) Task=`f88a5e16-a561-4c43-8be3-e0be25b90c4a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:07:16,934::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:16,934::task::1191::Storage.TaskManager.Task::(prepare) Task=`f88a5e16-a561-4c43-8be3-e0be25b90c4a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:16,934::task::595::Storage.TaskManager.Task::(_updateState) Task=`f88a5e16-a561-4c43-8be3-e0be25b90c4a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:07:16,935::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:07:16,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:07:16,935::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:07:16,936::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:07:16,936::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:07:16,936::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:07:16,936::task::993::Storage.TaskManager.Task::(_decref) Task=`f88a5e16-a561-4c43-8be3-e0be25b90c4a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:07:17,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:17,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130945 s, 78.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:19,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:07:19,706::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Dummy-33::DEBUG::2014-08-11 15:07:19,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132363 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:19,708::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:19,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:19,710::__init__::232::IOProcess::(_processLogs) (6366) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:19,712::__init__::232::IOProcess::(_processLogs) DEBUG|(6366) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:19,714::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:07:19,740::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00539203 s, 84.4 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:19,744::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:19,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:19,746::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:19,747::__init__::232::IOProcess::(_processLogs) (6367) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:19,749::__init__::232::IOProcess::(_processLogs) DEBUG|(6367) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:19,751::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:19,753::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:19,754::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:19,755::__init__::232::IOProcess::(_processLogs) (6368) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:19,757::__init__::232::IOProcess::(_processLogs) DEBUG|(6368) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:19,759::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:19,760::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:19,762::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:19,763::__init__::232::IOProcess::(_processLogs) (6369) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:19,765::__init__::232::IOProcess::(_processLogs) DEBUG|(6369) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:19,767::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:19,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:19,770::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:19,771::__init__::232::IOProcess::(_processLogs) (6370) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:19,773::__init__::232::IOProcess::(_processLogs) DEBUG|(6370) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:19,871::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:19,872::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:19,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:21,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:21,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133393 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:23,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:23,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131903 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:24,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:24,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:24,882::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:07:25,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`33539c25-ee56-40c5-b0a6-0cefdae25c9b`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:25,151::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:07:25,152::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00539203', 'lastCheck': '5.4', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:07:25,152::task::1191::Storage.TaskManager.Task::(prepare) Task=`33539c25-ee56-40c5-b0a6-0cefdae25c9b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00539203', 'lastCheck': '5.4', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:07:25,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`33539c25-ee56-40c5-b0a6-0cefdae25c9b`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:25,152::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:25,153::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:25,153::task::993::Storage.TaskManager.Task::(_decref) Task=`33539c25-ee56-40c5-b0a6-0cefdae25c9b`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:07:25,172::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:25,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:25,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182235 s, 56.2 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:07:26,948::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:07:26,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e3857ca-75a7-4343-81f1-04d6e5155163`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:26,949::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:07:26,962::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:26,963::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e3857ca-75a7-4343-81f1-04d6e5155163`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:26,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e3857ca-75a7-4343-81f1-04d6e5155163`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:26,963::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:26,963::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:26,963::task::993::Storage.TaskManager.Task::(_decref) Task=`6e3857ca-75a7-4343-81f1-04d6e5155163`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:07:26,974::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:07:26,974::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3e19010-7738-4d45-ba4b-da5bef5974e9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:07:26,975::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:07:26,976::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5a07bfce-4c10-463c-822b-f34f9605fbcf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:07:26,976::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:07:26,976::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:07:26,976::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5a07bfce-4c10-463c-822b-f34f9605fbcf`::Granted request >Thread-34::DEBUG::2014-08-11 15:07:26,977::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b3e19010-7738-4d45-ba4b-da5bef5974e9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:07:26,977::task::993::Storage.TaskManager.Task::(_decref) Task=`b3e19010-7738-4d45-ba4b-da5bef5974e9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:07:26,997::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:26,998::task::1191::Storage.TaskManager.Task::(prepare) Task=`b3e19010-7738-4d45-ba4b-da5bef5974e9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:26,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3e19010-7738-4d45-ba4b-da5bef5974e9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:07:26,998::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:07:26,998::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:07:26,999::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:07:26,999::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:07:26,999::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:07:26,999::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:07:27,000::task::993::Storage.TaskManager.Task::(_decref) Task=`b3e19010-7738-4d45-ba4b-da5bef5974e9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:07:27,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:28,003::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131504 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:29,775::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:29,804::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:29,805::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:07:29,805::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:07:29,807::__init__::232::IOProcess::(_processLogs) (6371) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:29,825::__init__::232::IOProcess::(_processLogs) (6371) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:29,826::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000600509 s, 758 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:29,827::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:29,827::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:29,828::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:29,828::__init__::232::IOProcess::(_processLogs) (6372) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:29,829::__init__::232::IOProcess::(_processLogs) DEBUG|(6372) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:29,830::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:29,831::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:29,831::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:29,832::__init__::232::IOProcess::(_processLogs) (6373) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:29,832::__init__::232::IOProcess::(_processLogs) (6373) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:29,832::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:29,832::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:29,833::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:29,833::__init__::232::IOProcess::(_processLogs) (6374) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:29,833::__init__::232::IOProcess::(_processLogs) DEBUG|(6374) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:29,834::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:29,835::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:29,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:29,836::__init__::232::IOProcess::(_processLogs) (6375) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:29,837::__init__::232::IOProcess::(_processLogs) (6375) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:29,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:29,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:29,904::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:30,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:30,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133104 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:32,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:32,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279739 s, 36.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:34,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:34,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130812 s, 78.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:34,908::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:34,910::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:35,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:36,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:36,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134569 s, 76.1 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:07:37,030::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:07:37,031::task::595::Storage.TaskManager.Task::(_updateState) Task=`aca362ea-ac03-4d89-a372-962b9b60ec0c`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:37,031::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:07:37,044::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:37,044::task::1191::Storage.TaskManager.Task::(prepare) Task=`aca362ea-ac03-4d89-a372-962b9b60ec0c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:37,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`aca362ea-ac03-4d89-a372-962b9b60ec0c`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:37,045::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:37,045::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:37,045::task::993::Storage.TaskManager.Task::(_decref) Task=`aca362ea-ac03-4d89-a372-962b9b60ec0c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:07:37,065::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:07:37,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b48e06d-84f0-4bdd-9194-328a6d0a05fc`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:07:37,069::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:07:37,071::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`488d3c8f-0783-456f-81ed-3355df4c1480`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:07:37,071::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:07:37,071::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:07:37,072::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`488d3c8f-0783-456f-81ed-3355df4c1480`::Granted request >Thread-34::DEBUG::2014-08-11 15:07:37,072::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7b48e06d-84f0-4bdd-9194-328a6d0a05fc`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:07:37,072::task::993::Storage.TaskManager.Task::(_decref) Task=`7b48e06d-84f0-4bdd-9194-328a6d0a05fc`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:07:37,093::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:37,093::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b48e06d-84f0-4bdd-9194-328a6d0a05fc`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:37,094::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b48e06d-84f0-4bdd-9194-328a6d0a05fc`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:07:37,094::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:07:37,094::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:07:37,094::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:07:37,095::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:07:37,095::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:07:37,095::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:07:37,095::task::993::Storage.TaskManager.Task::(_decref) Task=`7b48e06d-84f0-4bdd-9194-328a6d0a05fc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:07:38,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:38,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013149 s, 77.9 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:07:39,836::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:07:39,838::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,839::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,839::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,840::__init__::232::IOProcess::(_processLogs) DEBUG|(6376) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:07:39,841::__init__::232::IOProcess::(_processLogs) (6376) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:39,848::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:07:39,849::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,849::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,849::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,849::__init__::232::IOProcess::(_processLogs) DEBUG|(6377) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:07:39,850::__init__::232::IOProcess::(_processLogs) DEBUG|(6377) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:39,851::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:07:39,851::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,852::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,852::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,852::__init__::232::IOProcess::(_processLogs) (6378) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:07:39,853::__init__::232::IOProcess::(_processLogs) DEBUG|(6378) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:39,853::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:07:39,855::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,856::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,856::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,856::__init__::232::IOProcess::(_processLogs) (6379) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:39,857::__init__::232::IOProcess::(_processLogs) (6379) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:39,857::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,857::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,857::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,858::__init__::232::IOProcess::(_processLogs) (6380) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:07:39,858::__init__::232::IOProcess::(_processLogs) DEBUG|(6380) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:39,858::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:07:39,859::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:07:39,860::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:07:39,860::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,861::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,861::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:07:39,862::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:07:39,863::__init__::232::IOProcess::(_processLogs) (6381) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:39,880::__init__::232::IOProcess::(_processLogs) (6381) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:39,881::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000555066 s, 820 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:39,882::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,883::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,883::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,883::__init__::232::IOProcess::(_processLogs) (6382) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:39,884::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,885::__init__::232::IOProcess::(_processLogs) (6383) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:39,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,886::__init__::232::IOProcess::(_processLogs) (6384) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:39,886::__init__::232::IOProcess::(_processLogs) (6384) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:39,887::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:39,888::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:39,888::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:39,889::__init__::232::IOProcess::(_processLogs) (6385) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:39,889::__init__::232::IOProcess::(_processLogs) (6385) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:40,011::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:40,013::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:40,104::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:07:40,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea9041bc-253d-4950-a531-3fe9e7af7508`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:40,260::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:07:40,260::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000555066', 'lastCheck': '0.4', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:07:40,260::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea9041bc-253d-4950-a531-3fe9e7af7508`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000555066', 'lastCheck': '0.4', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:07:40,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea9041bc-253d-4950-a531-3fe9e7af7508`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:40,261::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:40,261::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:40,261::task::993::Storage.TaskManager.Task::(_decref) Task=`ea9041bc-253d-4950-a531-3fe9e7af7508`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:07:40,283::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:40,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:40,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133028 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:42,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:42,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131808 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:44,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:44,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136707 s, 74.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:45,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:45,113::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:45,204::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:46,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:46,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131481 s, 77.9 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:07:47,122::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:07:47,123::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2cdd22f-dc29-4c79-b3b7-82d9ee51ce73`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:47,123::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:07:47,136::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:47,136::task::1191::Storage.TaskManager.Task::(prepare) Task=`c2cdd22f-dc29-4c79-b3b7-82d9ee51ce73`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:47,137::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2cdd22f-dc29-4c79-b3b7-82d9ee51ce73`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:47,137::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:47,137::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:47,137::task::993::Storage.TaskManager.Task::(_decref) Task=`c2cdd22f-dc29-4c79-b3b7-82d9ee51ce73`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:07:47,147::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:07:47,148::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc697542-02fe-4cfe-96f4-11e6c8d6a89e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:07:47,148::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:07:47,149::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`980a4803-2568-46ff-8f19-b8b1b9a835a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:07:47,149::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:07:47,150::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:07:47,150::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`980a4803-2568-46ff-8f19-b8b1b9a835a9`::Granted request >Thread-34::DEBUG::2014-08-11 15:07:47,151::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cc697542-02fe-4cfe-96f4-11e6c8d6a89e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:07:47,151::task::993::Storage.TaskManager.Task::(_decref) Task=`cc697542-02fe-4cfe-96f4-11e6c8d6a89e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:07:47,171::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:47,171::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc697542-02fe-4cfe-96f4-11e6c8d6a89e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:47,171::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc697542-02fe-4cfe-96f4-11e6c8d6a89e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:07:47,171::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:07:47,172::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:07:47,172::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:07:47,172::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:07:47,173::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:07:47,173::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:07:47,173::task::993::Storage.TaskManager.Task::(_decref) Task=`cc697542-02fe-4cfe-96f4-11e6c8d6a89e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:07:48,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:48,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131608 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:49,890::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:49,891::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:49,892::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:07:49,892::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:07:49,893::__init__::232::IOProcess::(_processLogs) (6386) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:49,911::__init__::232::IOProcess::(_processLogs) (6386) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:49,913::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000713116 s, 638 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:49,914::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:49,914::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:49,915::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:49,915::__init__::232::IOProcess::(_processLogs) (6387) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:49,916::__init__::232::IOProcess::(_processLogs) DEBUG|(6387) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:49,917::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:49,918::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:49,918::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:49,918::__init__::232::IOProcess::(_processLogs) (6388) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:49,919::__init__::232::IOProcess::(_processLogs) (6388) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:49,919::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:49,920::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:49,920::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:49,920::__init__::232::IOProcess::(_processLogs) (6389) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:49,921::__init__::232::IOProcess::(_processLogs) (6389) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:49,921::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:49,922::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:49,922::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:49,923::__init__::232::IOProcess::(_processLogs) (6390) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:49,923::__init__::232::IOProcess::(_processLogs) (6390) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:50,211::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:50,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:50,304::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:50,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:50,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132423 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:52,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:52,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131439 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:07:54,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:54,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131693 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:55,310::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:55,311::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:07:55,371::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac2448fd-bb11-4b57-bbb2-98e3f790052f`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:55,404::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:07:55,406::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713116', 'lastCheck': '5.5', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:07:55,408::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac2448fd-bb11-4b57-bbb2-98e3f790052f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000713116', 'lastCheck': '5.5', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:07:55,408::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:07:55,409::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac2448fd-bb11-4b57-bbb2-98e3f790052f`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:55,411::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:55,412::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:55,413::task::993::Storage.TaskManager.Task::(_decref) Task=`ac2448fd-bb11-4b57-bbb2-98e3f790052f`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:07:55,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:07:56,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:57,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013111 s, 78.1 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:07:57,187::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:07:57,188::task::595::Storage.TaskManager.Task::(_updateState) Task=`2731c43c-8990-420f-b171-23b30211cfb1`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:07:57,188::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:07:57,201::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:57,201::task::1191::Storage.TaskManager.Task::(prepare) Task=`2731c43c-8990-420f-b171-23b30211cfb1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:07:57,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`2731c43c-8990-420f-b171-23b30211cfb1`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:07:57,202::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:07:57,202::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:07:57,202::task::993::Storage.TaskManager.Task::(_decref) Task=`2731c43c-8990-420f-b171-23b30211cfb1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:07:57,213::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:07:57,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`f451fa97-48ed-46b6-8a78-afd019a2e569`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:07:57,214::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:07:57,215::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8ca5c084-c351-42c5-a054-c8bf1f41f7da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:07:57,215::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:07:57,215::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:07:57,216::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8ca5c084-c351-42c5-a054-c8bf1f41f7da`::Granted request >Thread-34::DEBUG::2014-08-11 15:07:57,216::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f451fa97-48ed-46b6-8a78-afd019a2e569`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:07:57,216::task::993::Storage.TaskManager.Task::(_decref) Task=`f451fa97-48ed-46b6-8a78-afd019a2e569`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:07:57,236::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:57,237::task::1191::Storage.TaskManager.Task::(prepare) Task=`f451fa97-48ed-46b6-8a78-afd019a2e569`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:07:57,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`f451fa97-48ed-46b6-8a78-afd019a2e569`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:07:57,237::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:07:57,237::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:07:57,238::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:07:57,238::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:07:57,238::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:07:57,239::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:07:57,239::task::993::Storage.TaskManager.Task::(_decref) Task=`f451fa97-48ed-46b6-8a78-afd019a2e569`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:07:59,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:07:59,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132379 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:59,924::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:59,924::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:59,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:59,925::__init__::232::IOProcess::(_processLogs) (6391) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:59,926::__init__::232::IOProcess::(_processLogs) DEBUG|(6391) Queuing response >Thread-16::DEBUG::2014-08-11 15:07:59,927::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:07:59,945::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000444646 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:07:59,946::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:59,947::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:59,947::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:59,947::__init__::232::IOProcess::(_processLogs) (6392) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:07:59,948::__init__::232::IOProcess::(_processLogs) DEBUG|(6392) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:59,949::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:59,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:59,950::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:59,951::__init__::232::IOProcess::(_processLogs) (6393) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:59,951::__init__::232::IOProcess::(_processLogs) (6393) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:59,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:59,952::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:59,953::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:59,953::__init__::232::IOProcess::(_processLogs) (6394) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:59,953::__init__::232::IOProcess::(_processLogs) (6394) Queuing response >Thread-14::DEBUG::2014-08-11 15:07:59,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:07:59,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:07:59,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:07:59,956::__init__::232::IOProcess::(_processLogs) (6395) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:07:59,956::__init__::232::IOProcess::(_processLogs) (6395) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:00,419::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:00,420::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:00,504::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:01,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:01,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137898 s, 74.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:03,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:03,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133036 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:05,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:05,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131657 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:05,507::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:05,509::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:05,604::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:07,250::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:08:07,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6624d72-a989-4d21-829a-a2e72ceb18e3`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:07,251::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 15:08:07,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-111::INFO::2014-08-11 15:08:07,280::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:07,281::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6624d72-a989-4d21-829a-a2e72ceb18e3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:07,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6624d72-a989-4d21-829a-a2e72ceb18e3`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:07,281::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:07,281::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:07,282::task::993::Storage.TaskManager.Task::(_decref) Task=`b6624d72-a989-4d21-829a-a2e72ceb18e3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:08:07,294::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:08:07,295::task::595::Storage.TaskManager.Task::(_updateState) Task=`1fbe62b3-1562-4f26-a3f4-86a4b7ab7f14`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:08:07,295::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:08:07,296::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1d629cd4-4da8-44ea-8f37-9a68906524e2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:08:07,296::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:08:07,298::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:08:07,299::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1d629cd4-4da8-44ea-8f37-9a68906524e2`::Granted request >Thread-34::DEBUG::2014-08-11 15:08:07,302::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1fbe62b3-1562-4f26-a3f4-86a4b7ab7f14`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:08:07,303::task::993::Storage.TaskManager.Task::(_decref) Task=`1fbe62b3-1562-4f26-a3f4-86a4b7ab7f14`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:07,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0237927 s, 43.0 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 15:08:07,364::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:07,364::task::1191::Storage.TaskManager.Task::(prepare) Task=`1fbe62b3-1562-4f26-a3f4-86a4b7ab7f14`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:07,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`1fbe62b3-1562-4f26-a3f4-86a4b7ab7f14`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:08:07,365::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:08:07,365::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:08:07,365::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:08:07,366::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:08:07,366::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:08:07,366::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:08:07,366::task::993::Storage.TaskManager.Task::(_decref) Task=`1fbe62b3-1562-4f26-a3f4-86a4b7ab7f14`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:09,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:09,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132235 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:09,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:10,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:08:10,005::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:08:10,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:10,024::__init__::232::IOProcess::(_processLogs) (6396) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:10,025::__init__::232::IOProcess::(_processLogs) (6396) Queuing response >Thread-16::DEBUG::2014-08-11 15:08:10,026::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000595056 s, 765 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:10,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:10,027::__init__::232::IOProcess::(_processLogs) (6397) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:10,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:10,029::__init__::232::IOProcess::(_processLogs) (6398) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:10,030::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:10,030::__init__::232::IOProcess::(_processLogs) (6399) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:10,031::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:10,031::__init__::232::IOProcess::(_processLogs) (6400) Queuing response >Thread-111::DEBUG::2014-08-11 15:08:10,537::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc603f6d-9b00-4a73-88ff-760f32033221`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:10,537::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:08:10,538::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595056', 'lastCheck': '0.5', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:08:10,538::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc603f6d-9b00-4a73-88ff-760f32033221`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000595056', 'lastCheck': '0.5', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:08:10,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc603f6d-9b00-4a73-88ff-760f32033221`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:10,604::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:10,604::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:10,604::task::993::Storage.TaskManager.Task::(_decref) Task=`bc603f6d-9b00-4a73-88ff-760f32033221`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:10,611::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:10,612::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:10,613::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:10,622::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:11,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:11,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013203 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:13,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:13,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01317 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:15,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:15,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132162 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:15,617::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:15,618::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:15,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 15:08:15,768::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d9089cc-6adf-49e6-9abd-77533bbf18ec`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:08:15,768::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:08:15,769::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:15,770::__init__::232::IOProcess::(_processLogs) (6401) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:08:15,771::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:15,772::__init__::232::IOProcess::(_processLogs) (6402) Queuing response >Thread-55::INFO::2014-08-11 15:08:15,772::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:08:15,772::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d9089cc-6adf-49e6-9abd-77533bbf18ec`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:08:15,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d9089cc-6adf-49e6-9abd-77533bbf18ec`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:08:15,773::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:08:15,773::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:08:15,773::task::993::Storage.TaskManager.Task::(_decref) Task=`9d9089cc-6adf-49e6-9abd-77533bbf18ec`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:08:17,378::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:08:17,379::task::595::Storage.TaskManager.Task::(_updateState) Task=`7163659a-30a9-4942-bba5-3e7ebf4c3159`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:17,379::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:08:17,393::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:17,393::task::1191::Storage.TaskManager.Task::(prepare) Task=`7163659a-30a9-4942-bba5-3e7ebf4c3159`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:17,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`7163659a-30a9-4942-bba5-3e7ebf4c3159`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:17,393::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:17,394::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:17,394::task::993::Storage.TaskManager.Task::(_decref) Task=`7163659a-30a9-4942-bba5-3e7ebf4c3159`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:08:17,404::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:08:17,405::task::595::Storage.TaskManager.Task::(_updateState) Task=`81ecb56d-3904-4415-b7a3-9d02abeccca0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:08:17,405::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:08:17,406::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`02f45896-ba19-4cfd-816f-803382473e12`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:08:17,406::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:08:17,406::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:08:17,406::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`02f45896-ba19-4cfd-816f-803382473e12`::Granted request >Thread-34::DEBUG::2014-08-11 15:08:17,407::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`81ecb56d-3904-4415-b7a3-9d02abeccca0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:08:17,407::task::993::Storage.TaskManager.Task::(_decref) Task=`81ecb56d-3904-4415-b7a3-9d02abeccca0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:08:17,427::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:17,427::task::1191::Storage.TaskManager.Task::(prepare) Task=`81ecb56d-3904-4415-b7a3-9d02abeccca0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:17,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`81ecb56d-3904-4415-b7a3-9d02abeccca0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:08:17,428::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:08:17,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:08:17,428::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:08:17,429::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:08:17,429::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:08:17,429::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:08:17,429::task::993::Storage.TaskManager.Task::(_decref) Task=`81ecb56d-3904-4415-b7a3-9d02abeccca0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:17,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:17,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132081 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:19,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:19,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132851 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:20,033::__init__::232::IOProcess::(_processLogs) (6403) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:08:20,034::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:08:20,053::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418251 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:20,054::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:20,055::__init__::232::IOProcess::(_processLogs) (6404) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:20,056::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:20,056::__init__::232::IOProcess::(_processLogs) (6405) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:20,057::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:20,058::__init__::232::IOProcess::(_processLogs) (6406) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:20,058::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:20,059::__init__::232::IOProcess::(_processLogs) (6407) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:20,059::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6407) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:20,621::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:20,623::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:20,704::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:21,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:21,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01336 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:23,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:23,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131757 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:25,707::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:25,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:25,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`393855b2-3efa-45fa-87f8-0588b14f3da4`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:25,804::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:08:25,806::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418251', 'lastCheck': '5.7', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:08:25,808::task::1191::Storage.TaskManager.Task::(prepare) Task=`393855b2-3efa-45fa-87f8-0588b14f3da4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418251', 'lastCheck': '5.7', 'valid': True}} >Dummy-33::DEBUG::2014-08-11 15:08:25,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-111::DEBUG::2014-08-11 15:08:25,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`393855b2-3efa-45fa-87f8-0588b14f3da4`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:25,827::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:25,829::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:25,831::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:25,833::task::993::Storage.TaskManager.Task::(_decref) Task=`393855b2-3efa-45fa-87f8-0588b14f3da4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:25,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0165881 s, 61.7 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:08:25,860::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:27,441::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:08:27,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a0b5909-b6a2-4664-9d08-fd2a8fe1b6a8`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:27,443::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:08:27,455::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:27,456::task::1191::Storage.TaskManager.Task::(prepare) Task=`8a0b5909-b6a2-4664-9d08-fd2a8fe1b6a8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:27,456::task::595::Storage.TaskManager.Task::(_updateState) Task=`8a0b5909-b6a2-4664-9d08-fd2a8fe1b6a8`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:27,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:27,456::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:27,457::task::993::Storage.TaskManager.Task::(_decref) Task=`8a0b5909-b6a2-4664-9d08-fd2a8fe1b6a8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:08:27,467::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:08:27,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`279d346c-e021-4b35-a9dc-d1c65527eb13`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:08:27,468::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:08:27,468::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0b611158-969b-44a3-9cfd-4ccf56f6acd1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:08:27,469::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:08:27,469::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:08:27,469::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0b611158-969b-44a3-9cfd-4ccf56f6acd1`::Granted request >Thread-34::DEBUG::2014-08-11 15:08:27,470::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`279d346c-e021-4b35-a9dc-d1c65527eb13`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:08:27,470::task::993::Storage.TaskManager.Task::(_decref) Task=`279d346c-e021-4b35-a9dc-d1c65527eb13`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:08:27,490::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:27,491::task::1191::Storage.TaskManager.Task::(prepare) Task=`279d346c-e021-4b35-a9dc-d1c65527eb13`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:27,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`279d346c-e021-4b35-a9dc-d1c65527eb13`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:08:27,491::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:08:27,491::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:08:27,492::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:08:27,492::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:08:27,492::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:08:27,492::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:08:27,493::task::993::Storage.TaskManager.Task::(_decref) Task=`279d346c-e021-4b35-a9dc-d1c65527eb13`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:27,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:27,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134192 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:29,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:29,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131407 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:30,061::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:30,061::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:30,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:30,062::__init__::232::IOProcess::(_processLogs) (6408) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:30,062::__init__::232::IOProcess::(_processLogs) (6408) Queuing response >Thread-16::DEBUG::2014-08-11 15:08:30,063::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:08:30,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000410671 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:30,084::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:30,084::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:30,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:30,085::__init__::232::IOProcess::(_processLogs) (6409) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:30,085::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6409) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:30,087::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:30,087::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:30,088::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:30,088::__init__::232::IOProcess::(_processLogs) (6410) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:30,089::__init__::232::IOProcess::(_processLogs) (6410) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:30,089::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:30,089::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:30,089::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:30,090::__init__::232::IOProcess::(_processLogs) (6411) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:30,090::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6411) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:30,091::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:30,092::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:30,092::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:30,093::__init__::232::IOProcess::(_processLogs) (6412) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:30,093::__init__::232::IOProcess::(_processLogs) (6412) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:30,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:30,842::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:30,843::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:31,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:32,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0286481 s, 35.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:34,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:34,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132031 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:35,850::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:35,852::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:35,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:36,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:36,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134744 s, 76.0 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:08:37,505::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:08:37,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b4b3259-10cf-422e-9c9d-0924cf5e8f98`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:37,507::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:08:37,522::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:37,522::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b4b3259-10cf-422e-9c9d-0924cf5e8f98`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:37,522::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b4b3259-10cf-422e-9c9d-0924cf5e8f98`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:37,523::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:37,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:37,523::task::993::Storage.TaskManager.Task::(_decref) Task=`4b4b3259-10cf-422e-9c9d-0924cf5e8f98`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:08:37,535::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:08:37,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b28868a-d8d8-4159-81e8-4b9185cd7a9e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:08:37,536::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:08:37,537::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1c23deb0-09cf-4389-9cff-5e187be5981c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:08:37,537::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:08:37,538::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:08:37,538::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1c23deb0-09cf-4389-9cff-5e187be5981c`::Granted request >Thread-34::DEBUG::2014-08-11 15:08:37,539::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9b28868a-d8d8-4159-81e8-4b9185cd7a9e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:08:37,539::task::993::Storage.TaskManager.Task::(_decref) Task=`9b28868a-d8d8-4159-81e8-4b9185cd7a9e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:08:37,558::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:37,559::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b28868a-d8d8-4159-81e8-4b9185cd7a9e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:37,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b28868a-d8d8-4159-81e8-4b9185cd7a9e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:08:37,559::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:08:37,559::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:08:37,560::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:08:37,560::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:08:37,560::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:08:37,560::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:08:37,561::task::993::Storage.TaskManager.Task::(_decref) Task=`9b28868a-d8d8-4159-81e8-4b9185cd7a9e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:38,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:38,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131638 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:40,094::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:40,095::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:40,096::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:08:40,096::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:08:40,097::__init__::232::IOProcess::(_processLogs) (6413) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:40,117::__init__::232::IOProcess::(_processLogs) (6413) Queuing response >Thread-16::DEBUG::2014-08-11 15:08:40,119::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00100735 s, 452 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:40,120::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:40,121::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:40,121::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:40,121::__init__::232::IOProcess::(_processLogs) (6414) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:40,122::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6414) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:40,123::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:40,124::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:40,125::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:40,125::__init__::232::IOProcess::(_processLogs) (6415) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:40,125::__init__::232::IOProcess::(_processLogs) (6415) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:40,126::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:40,126::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:40,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:40,127::__init__::232::IOProcess::(_processLogs) (6416) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:40,127::__init__::232::IOProcess::(_processLogs) (6416) Queuing response >Thread-14::DEBUG::2014-08-11 15:08:40,128::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:40,128::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:40,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:40,130::__init__::232::IOProcess::(_processLogs) (6417) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:40,130::__init__::232::IOProcess::(_processLogs) (6417) Queuing response >Dummy-33::DEBUG::2014-08-11 15:08:40,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:40,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131416 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:40,912::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:40,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:40,974::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3008ac6-a2cb-4b8a-92b3-55f195ab43e7`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:41,006::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:08:41,008::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00100735', 'lastCheck': '0.9', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:08:41,009::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3008ac6-a2cb-4b8a-92b3-55f195ab43e7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00100735', 'lastCheck': '0.9', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:08:41,010::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3008ac6-a2cb-4b8a-92b3-55f195ab43e7`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:41,010::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:41,011::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:41,012::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:41,013::task::993::Storage.TaskManager.Task::(_decref) Task=`f3008ac6-a2cb-4b8a-92b3-55f195ab43e7`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:08:41,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:42,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:42,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136598 s, 75.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:44,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:44,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133934 s, 76.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:46,018::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:46,019::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:46,105::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:46,568::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:46,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0287137 s, 35.7 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:08:47,574::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:08:47,575::task::595::Storage.TaskManager.Task::(_updateState) Task=`65e54095-b315-44db-b1ae-53f141dbd7a1`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:47,575::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:08:47,588::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:47,588::task::1191::Storage.TaskManager.Task::(prepare) Task=`65e54095-b315-44db-b1ae-53f141dbd7a1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:47,588::task::595::Storage.TaskManager.Task::(_updateState) Task=`65e54095-b315-44db-b1ae-53f141dbd7a1`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:47,589::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:47,589::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:47,589::task::993::Storage.TaskManager.Task::(_decref) Task=`65e54095-b315-44db-b1ae-53f141dbd7a1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:08:47,602::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:08:47,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`98a0561e-3317-4997-a404-e667989886a8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:08:47,603::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:08:47,603::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c6f7a1d4-5d2d-4e79-8faf-b6884db835b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:08:47,604::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:08:47,604::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:08:47,604::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c6f7a1d4-5d2d-4e79-8faf-b6884db835b3`::Granted request >Thread-34::DEBUG::2014-08-11 15:08:47,605::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`98a0561e-3317-4997-a404-e667989886a8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:08:47,605::task::993::Storage.TaskManager.Task::(_decref) Task=`98a0561e-3317-4997-a404-e667989886a8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:08:47,625::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:47,625::task::1191::Storage.TaskManager.Task::(prepare) Task=`98a0561e-3317-4997-a404-e667989886a8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136440832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:47,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`98a0561e-3317-4997-a404-e667989886a8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:08:47,626::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:08:47,626::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:08:47,626::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:08:47,627::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:08:47,627::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:08:47,627::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:08:47,627::task::993::Storage.TaskManager.Task::(_decref) Task=`98a0561e-3317-4997-a404-e667989886a8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:48,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:48,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132728 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:50,131::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:50,132::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:08:50,133::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:08:50,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:50,151::__init__::232::IOProcess::(_processLogs) (6418) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:50,152::__init__::232::IOProcess::(_processLogs) (6418) Queuing response >Thread-16::DEBUG::2014-08-11 15:08:50,153::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000968981 s, 470 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:08:50,154::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:08:50,154::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:08:50,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:50,155::__init__::232::IOProcess::(_processLogs) (6419) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:08:50,156::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:50,156::__init__::232::IOProcess::(_processLogs) (6420) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:50,157::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:50,158::__init__::232::IOProcess::(_processLogs) (6421) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:50,159::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:08:50,159::__init__::232::IOProcess::(_processLogs) (6422) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:08:50,159::__init__::232::IOProcess::(_processLogs) (6422) Queuing response >Dummy-33::DEBUG::2014-08-11 15:08:50,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:50,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132887 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:51,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:51,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:51,205::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:52,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:52,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133986 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:08:54,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:54,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132417 s, 77.3 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:08:56,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`0865f84b-58c6-4cc7-9307-b132c2b1ba12`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:56,133::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:08:56,205::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000968981', 'lastCheck': '6.0', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:08:56,205::task::1191::Storage.TaskManager.Task::(prepare) Task=`0865f84b-58c6-4cc7-9307-b132c2b1ba12`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000968981', 'lastCheck': '6.0', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:08:56,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`0865f84b-58c6-4cc7-9307-b132c2b1ba12`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:56,205::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:56,205::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:56,206::task::993::Storage.TaskManager.Task::(_decref) Task=`0865f84b-58c6-4cc7-9307-b132c2b1ba12`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:56,211::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:56,213::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:08:56,214::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:08:56,226::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:08:56,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:56,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134917 s, 75.9 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:08:57,639::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:08:57,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3332017-1eb0-4bb6-b2d5-a00c2c140be4`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:08:57,640::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:08:57,653::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:57,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`e3332017-1eb0-4bb6-b2d5-a00c2c140be4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:08:57,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`e3332017-1eb0-4bb6-b2d5-a00c2c140be4`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:08:57,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:08:57,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:08:57,654::task::993::Storage.TaskManager.Task::(_decref) Task=`e3332017-1eb0-4bb6-b2d5-a00c2c140be4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:08:57,666::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:08:57,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`1476daba-632c-4b20-b008-9cf1b71ea32f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:08:57,667::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:08:57,668::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`86dfd966-6404-44e9-9e0d-a755512d2415`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:08:57,668::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:08:57,668::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:08:57,669::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`86dfd966-6404-44e9-9e0d-a755512d2415`::Granted request >Thread-34::DEBUG::2014-08-11 15:08:57,669::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1476daba-632c-4b20-b008-9cf1b71ea32f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:08:57,669::task::993::Storage.TaskManager.Task::(_decref) Task=`1476daba-632c-4b20-b008-9cf1b71ea32f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:08:57,689::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136309760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:57,690::task::1191::Storage.TaskManager.Task::(prepare) Task=`1476daba-632c-4b20-b008-9cf1b71ea32f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136309760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:08:57,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`1476daba-632c-4b20-b008-9cf1b71ea32f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:08:57,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:08:57,690::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:08:57,691::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:08:57,691::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:08:57,691::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:08:57,691::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:08:57,692::task::993::Storage.TaskManager.Task::(_decref) Task=`1476daba-632c-4b20-b008-9cf1b71ea32f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:08:58,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:08:58,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013278 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:00,161::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:00,162::__init__::232::IOProcess::(_processLogs) (6423) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:09:00,162::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:09:00,181::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417807 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:00,182::__init__::232::IOProcess::(_processLogs) (6424) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:00,184::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:00,184::__init__::232::IOProcess::(_processLogs) (6425) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:00,185::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:00,185::__init__::232::IOProcess::(_processLogs) (6426) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:00,186::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:00,186::__init__::232::IOProcess::(_processLogs) (6427) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:00,187::__init__::232::IOProcess::(_processLogs) (6427) Queuing response >Dummy-33::DEBUG::2014-08-11 15:09:00,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:01,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157969 s, 64.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:01,221::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:01,222::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:01,305::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:03,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:03,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132537 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:05,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:05,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131815 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:06,310::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:06,312::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:06,313::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:07,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:07,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132252 s, 77.4 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:09:07,704::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:09:07,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8931952-c953-4c18-ae1b-2b8dd9b463c4`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:07,705::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:09:07,718::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:07,719::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8931952-c953-4c18-ae1b-2b8dd9b463c4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:07,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8931952-c953-4c18-ae1b-2b8dd9b463c4`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:07,719::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:07,719::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:07,720::task::993::Storage.TaskManager.Task::(_decref) Task=`a8931952-c953-4c18-ae1b-2b8dd9b463c4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:09:07,729::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:09:07,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`40e38aae-0c4c-49a2-a991-d3ebfa61387a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:09:07,730::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:09:07,731::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a8ee1d59-c4f9-4ff9-a276-75af6189b3da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:09:07,731::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:09:07,732::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:09:07,732::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a8ee1d59-c4f9-4ff9-a276-75af6189b3da`::Granted request >Thread-34::DEBUG::2014-08-11 15:09:07,733::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`40e38aae-0c4c-49a2-a991-d3ebfa61387a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:09:07,733::task::993::Storage.TaskManager.Task::(_decref) Task=`40e38aae-0c4c-49a2-a991-d3ebfa61387a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:09:07,753::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136244224', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:07,753::task::1191::Storage.TaskManager.Task::(prepare) Task=`40e38aae-0c4c-49a2-a991-d3ebfa61387a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136244224', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:07,753::task::595::Storage.TaskManager.Task::(_updateState) Task=`40e38aae-0c4c-49a2-a991-d3ebfa61387a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:09:07,753::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:09:07,754::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:09:07,754::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:09:07,754::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:09:07,754::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:09:07,755::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:09:07,755::task::993::Storage.TaskManager.Task::(_decref) Task=`40e38aae-0c4c-49a2-a991-d3ebfa61387a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:09:09,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:09,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132235 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:10,189::__init__::232::IOProcess::(_processLogs) (6428) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:09:10,190::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:09:10,208::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416368 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:10,210::__init__::232::IOProcess::(_processLogs) (6429) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:10,211::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:10,212::__init__::232::IOProcess::(_processLogs) (6430) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:10,213::__init__::232::IOProcess::(_processLogs) (6430) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:10,213::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:10,214::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:10,214::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:10,214::__init__::232::IOProcess::(_processLogs) (6431) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:10,215::__init__::232::IOProcess::(_processLogs) (6431) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:10,215::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:10,216::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:10,216::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:10,217::__init__::232::IOProcess::(_processLogs) (6432) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:10,217::__init__::232::IOProcess::(_processLogs) (6432) Queuing response >Dummy-33::DEBUG::2014-08-11 15:09:11,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:11,295::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0146743 s, 69.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:11,319::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:11,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:11,330::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3795974-0c56-4835-b757-9aba858df844`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:11,407::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:09:11,409::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416368', 'lastCheck': '1.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:11,410::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:11,411::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3795974-0c56-4835-b757-9aba858df844`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416368', 'lastCheck': '1.2', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:09:11,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3795974-0c56-4835-b757-9aba858df844`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:11,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:11,414::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:11,415::task::993::Storage.TaskManager.Task::(_decref) Task=`a3795974-0c56-4835-b757-9aba858df844`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:09:11,436::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:13,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:13,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132524 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:15,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:15,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132032 s, 77.6 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:09:15,888::task::595::Storage.TaskManager.Task::(_updateState) Task=`80d7a63d-e947-44dd-8b59-ab34799c76b3`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:09:15,888::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:09:15,889::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:15,890::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:15,890::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:15,890::__init__::232::IOProcess::(_processLogs) (6433) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:09:15,893::__init__::232::IOProcess::(_processLogs) DEBUG|(6433) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:15,894::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:15,894::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:15,894::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:15,894::__init__::232::IOProcess::(_processLogs) (6434) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:09:15,895::__init__::232::IOProcess::(_processLogs) DEBUG|(6434) Queuing response >Thread-55::INFO::2014-08-11 15:09:15,895::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:09:15,896::task::1191::Storage.TaskManager.Task::(prepare) Task=`80d7a63d-e947-44dd-8b59-ab34799c76b3`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:09:15,896::task::595::Storage.TaskManager.Task::(_updateState) Task=`80d7a63d-e947-44dd-8b59-ab34799c76b3`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:09:15,896::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:09:15,896::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:09:15,896::task::993::Storage.TaskManager.Task::(_decref) Task=`80d7a63d-e947-44dd-8b59-ab34799c76b3`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:16,420::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:16,422::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:16,506::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:17,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:17,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132623 s, 77.2 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:09:17,766::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:09:17,767::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e359a61-0663-4bdb-9caf-3651eadb11bf`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:17,767::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:09:17,780::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:17,781::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e359a61-0663-4bdb-9caf-3651eadb11bf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:17,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e359a61-0663-4bdb-9caf-3651eadb11bf`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:17,781::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:17,781::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:17,782::task::993::Storage.TaskManager.Task::(_decref) Task=`5e359a61-0663-4bdb-9caf-3651eadb11bf`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:09:17,792::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:09:17,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`1906f3b5-04bc-4989-8e8a-07104ee087ce`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:09:17,793::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:09:17,794::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6cf975a2-b194-4d3d-8997-39af25cb53dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:09:17,794::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:09:17,795::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:09:17,795::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6cf975a2-b194-4d3d-8997-39af25cb53dd`::Granted request >Thread-34::DEBUG::2014-08-11 15:09:17,795::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1906f3b5-04bc-4989-8e8a-07104ee087ce`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:09:17,796::task::993::Storage.TaskManager.Task::(_decref) Task=`1906f3b5-04bc-4989-8e8a-07104ee087ce`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:09:17,815::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:17,816::task::1191::Storage.TaskManager.Task::(prepare) Task=`1906f3b5-04bc-4989-8e8a-07104ee087ce`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:17,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`1906f3b5-04bc-4989-8e8a-07104ee087ce`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:09:17,816::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:09:17,816::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:09:17,817::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:09:17,817::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:09:17,817::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:09:17,818::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:09:17,818::task::993::Storage.TaskManager.Task::(_decref) Task=`1906f3b5-04bc-4989-8e8a-07104ee087ce`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:09:19,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:19,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218463 s, 46.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:20,218::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:20,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:20,218::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:20,219::__init__::232::IOProcess::(_processLogs) (6435) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:20,219::__init__::232::IOProcess::(_processLogs) DEBUG|(6435) Queuing response >Thread-16::DEBUG::2014-08-11 15:09:20,221::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:09:20,240::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416603 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:20,241::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:20,241::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:20,242::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:20,242::__init__::232::IOProcess::(_processLogs) (6436) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:20,243::__init__::232::IOProcess::(_processLogs) DEBUG|(6436) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:20,244::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:20,244::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:20,245::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:20,245::__init__::232::IOProcess::(_processLogs) (6437) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:20,246::__init__::232::IOProcess::(_processLogs) DEBUG|(6437) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:20,246::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:20,247::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:20,247::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:20,247::__init__::232::IOProcess::(_processLogs) (6438) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:20,248::__init__::232::IOProcess::(_processLogs) DEBUG|(6438) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:20,249::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:20,249::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:20,249::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:20,250::__init__::232::IOProcess::(_processLogs) (6439) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:20,250::__init__::232::IOProcess::(_processLogs) DEBUG|(6439) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:21,513::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:21,515::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:21,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:21,625::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:21,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133376 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:23,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:23,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132157 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:25,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:25,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132198 s, 77.5 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:09:26,528::task::595::Storage.TaskManager.Task::(_updateState) Task=`08ca0d1c-afbc-4b65-b0a5-3a812b78195f`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:26,528::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:09:26,606::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416603', 'lastCheck': '6.4', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:09:26,606::task::1191::Storage.TaskManager.Task::(prepare) Task=`08ca0d1c-afbc-4b65-b0a5-3a812b78195f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416603', 'lastCheck': '6.4', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:09:26,606::task::595::Storage.TaskManager.Task::(_updateState) Task=`08ca0d1c-afbc-4b65-b0a5-3a812b78195f`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:26,606::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:26,607::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:26,607::task::993::Storage.TaskManager.Task::(_decref) Task=`08ca0d1c-afbc-4b65-b0a5-3a812b78195f`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:09:26,626::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:26,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:26,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:26,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:27,771::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:27,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132643 s, 77.2 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:09:27,835::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:09:27,836::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a6e0370-680f-4eb7-ae7c-8a65912d53ba`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:27,836::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:09:27,849::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:27,849::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a6e0370-680f-4eb7-ae7c-8a65912d53ba`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:27,849::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a6e0370-680f-4eb7-ae7c-8a65912d53ba`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:27,850::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:27,850::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:27,850::task::993::Storage.TaskManager.Task::(_decref) Task=`7a6e0370-680f-4eb7-ae7c-8a65912d53ba`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:09:27,860::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:09:27,861::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a7c8bf8-98e8-4a37-9e14-78f93d6ea1d0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:09:27,861::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:09:27,862::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a04ec1de-5f6a-402e-90a2-e6aa8eb555dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:09:27,862::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:09:27,862::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:09:27,863::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a04ec1de-5f6a-402e-90a2-e6aa8eb555dd`::Granted request >Thread-34::DEBUG::2014-08-11 15:09:27,863::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2a7c8bf8-98e8-4a37-9e14-78f93d6ea1d0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:09:27,863::task::993::Storage.TaskManager.Task::(_decref) Task=`2a7c8bf8-98e8-4a37-9e14-78f93d6ea1d0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:09:27,883::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:27,884::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a7c8bf8-98e8-4a37-9e14-78f93d6ea1d0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:27,884::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a7c8bf8-98e8-4a37-9e14-78f93d6ea1d0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:09:27,884::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:09:27,884::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:09:27,885::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:09:27,885::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:09:27,885::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:09:27,885::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:09:27,886::task::993::Storage.TaskManager.Task::(_decref) Task=`2a7c8bf8-98e8-4a37-9e14-78f93d6ea1d0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:09:29,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:29,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132186 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:30,252::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:30,253::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:09:30,254::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:09:30,255::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:30,273::__init__::232::IOProcess::(_processLogs) (6440) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:30,273::__init__::232::IOProcess::(_processLogs) (6440) Queuing response >Thread-16::DEBUG::2014-08-11 15:09:30,274::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000871802 s, 522 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:30,275::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:30,276::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:30,276::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:30,276::__init__::232::IOProcess::(_processLogs) (6441) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:30,278::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:30,278::__init__::232::IOProcess::(_processLogs) (6442) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:30,278::__init__::232::IOProcess::(_processLogs) DEBUG|(6442) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:30,279::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:30,280::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:30,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:30,280::__init__::232::IOProcess::(_processLogs) (6443) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:30,281::__init__::232::IOProcess::(_processLogs) DEBUG|(6443) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:30,282::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:30,282::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:30,283::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:30,284::__init__::232::IOProcess::(_processLogs) (6444) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:30,284::__init__::232::IOProcess::(_processLogs) (6444) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:31,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:31,644::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:31,706::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:31,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:31,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133643 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:33,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:33,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131214 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:35,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:36,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132532 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:36,714::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:36,715::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:36,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:37,930::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:09:37,930::task::595::Storage.TaskManager.Task::(_updateState) Task=`995ad4e3-41cd-448a-b469-dbb65644321b`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:37,931::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:09:37,943::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:37,944::task::1191::Storage.TaskManager.Task::(prepare) Task=`995ad4e3-41cd-448a-b469-dbb65644321b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:37,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`995ad4e3-41cd-448a-b469-dbb65644321b`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:37,944::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:37,944::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:37,945::task::993::Storage.TaskManager.Task::(_decref) Task=`995ad4e3-41cd-448a-b469-dbb65644321b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:09:37,955::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:09:37,956::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf6b8a7-93ec-479f-9971-92285e3494df`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:09:37,956::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:09:37,957::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2a397878-4a2a-46e7-96d7-3c3d6c6c8844`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:09:37,957::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:09:37,957::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:09:37,957::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2a397878-4a2a-46e7-96d7-3c3d6c6c8844`::Granted request >Thread-34::DEBUG::2014-08-11 15:09:37,958::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`daf6b8a7-93ec-479f-9971-92285e3494df`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:09:37,958::task::993::Storage.TaskManager.Task::(_decref) Task=`daf6b8a7-93ec-479f-9971-92285e3494df`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:09:37,979::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:37,979::task::1191::Storage.TaskManager.Task::(prepare) Task=`daf6b8a7-93ec-479f-9971-92285e3494df`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:37,979::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf6b8a7-93ec-479f-9971-92285e3494df`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:09:37,980::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:09:37,980::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:09:37,980::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:09:37,981::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:09:37,981::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:09:37,981::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:09:37,981::task::993::Storage.TaskManager.Task::(_decref) Task=`daf6b8a7-93ec-479f-9971-92285e3494df`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:09:38,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:38,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132611 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:40,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:40,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132807 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:40,284::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:40,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:40,286::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:09:40,287::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:09:40,287::__init__::232::IOProcess::(_processLogs) (6445) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:40,305::__init__::232::IOProcess::(_processLogs) (6445) Queuing response >Thread-16::DEBUG::2014-08-11 15:09:40,307::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000461288 s, 986 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:40,308::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:40,308::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:40,309::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:40,309::__init__::232::IOProcess::(_processLogs) (6446) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:40,310::__init__::232::IOProcess::(_processLogs) DEBUG|(6446) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:40,311::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:40,311::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:40,311::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:40,312::__init__::232::IOProcess::(_processLogs) (6447) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:40,312::__init__::232::IOProcess::(_processLogs) DEBUG|(6447) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:40,313::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:40,313::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:40,314::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:40,314::__init__::232::IOProcess::(_processLogs) (6448) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:40,315::__init__::232::IOProcess::(_processLogs) DEBUG|(6448) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:40,315::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:40,316::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:40,316::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:40,316::__init__::232::IOProcess::(_processLogs) (6449) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:40,317::__init__::232::IOProcess::(_processLogs) DEBUG|(6449) Queuing response >Thread-111::DEBUG::2014-08-11 15:09:41,718::task::595::Storage.TaskManager.Task::(_updateState) Task=`203bf4ef-90bf-4819-9fb5-ce07d878baf9`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:41,719::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:41,723::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::INFO::2014-08-11 15:09:41,807::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461288', 'lastCheck': '1.5', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:09:41,808::task::1191::Storage.TaskManager.Task::(prepare) Task=`203bf4ef-90bf-4819-9fb5-ce07d878baf9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000461288', 'lastCheck': '1.5', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:09:41,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`203bf4ef-90bf-4819-9fb5-ce07d878baf9`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:41,811::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:41,811::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:41,812::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:41,813::task::993::Storage.TaskManager.Task::(_decref) Task=`203bf4ef-90bf-4819-9fb5-ce07d878baf9`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:41,817::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:41,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:42,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:42,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131653 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:44,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:44,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013195 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:46,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:46,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131706 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:46,828::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:46,829::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:46,830::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:47,994::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:09:47,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`905ac9ac-f818-4e73-8a1e-5a5c5cda32b6`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:47,995::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:09:48,008::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:48,008::task::1191::Storage.TaskManager.Task::(prepare) Task=`905ac9ac-f818-4e73-8a1e-5a5c5cda32b6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:48,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`905ac9ac-f818-4e73-8a1e-5a5c5cda32b6`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:48,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:48,009::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:48,009::task::993::Storage.TaskManager.Task::(_decref) Task=`905ac9ac-f818-4e73-8a1e-5a5c5cda32b6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:09:48,019::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:09:48,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`a759dcdb-8408-4da9-ab9d-c856a95d3bae`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:09:48,020::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:09:48,020::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2fd518b8-ac28-45da-bd00-9d7943ff4f5d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:09:48,021::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:09:48,021::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:09:48,021::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2fd518b8-ac28-45da-bd00-9d7943ff4f5d`::Granted request >Thread-34::DEBUG::2014-08-11 15:09:48,022::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a759dcdb-8408-4da9-ab9d-c856a95d3bae`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:09:48,022::task::993::Storage.TaskManager.Task::(_decref) Task=`a759dcdb-8408-4da9-ab9d-c856a95d3bae`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:09:48,042::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:48,042::task::1191::Storage.TaskManager.Task::(prepare) Task=`a759dcdb-8408-4da9-ab9d-c856a95d3bae`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:48,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`a759dcdb-8408-4da9-ab9d-c856a95d3bae`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:09:48,043::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:09:48,043::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:09:48,043::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:09:48,044::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:09:48,044::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:09:48,044::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:09:48,044::task::993::Storage.TaskManager.Task::(_decref) Task=`a759dcdb-8408-4da9-ab9d-c856a95d3bae`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:09:48,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:48,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132911 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:50,319::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:50,319::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:50,319::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:50,320::__init__::232::IOProcess::(_processLogs) (6450) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:50,320::__init__::232::IOProcess::(_processLogs) DEBUG|(6450) Queuing response >Thread-16::DEBUG::2014-08-11 15:09:50,322::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:09:50,340::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417751 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:09:50,341::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:50,341::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:50,342::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:50,342::__init__::232::IOProcess::(_processLogs) (6451) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:09:50,343::__init__::232::IOProcess::(_processLogs) DEBUG|(6451) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:50,344::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:50,344::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:50,345::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:50,345::__init__::232::IOProcess::(_processLogs) (6452) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:50,346::__init__::232::IOProcess::(_processLogs) DEBUG|(6452) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:50,346::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:50,347::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:50,347::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:50,347::__init__::232::IOProcess::(_processLogs) (6453) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:50,348::__init__::232::IOProcess::(_processLogs) DEBUG|(6453) Queuing response >Thread-14::DEBUG::2014-08-11 15:09:50,349::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:09:50,349::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:09:50,349::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:09:50,349::__init__::232::IOProcess::(_processLogs) (6454) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:09:50,350::__init__::232::IOProcess::(_processLogs) DEBUG|(6454) Queuing response >Dummy-33::DEBUG::2014-08-11 15:09:50,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:50,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132632 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:51,837::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:51,839::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:51,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:09:52,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:52,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132444 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:54,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:54,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0166336 s, 61.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:09:56,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:56,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132525 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:56,910::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:56,911::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:56,930::task::595::Storage.TaskManager.Task::(_updateState) Task=`d69fbd86-71a1-4f33-a7a3-998d7267d4c7`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:57,007::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:09:57,009::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417751', 'lastCheck': '6.7', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:09:57,010::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:57,011::task::1191::Storage.TaskManager.Task::(prepare) Task=`d69fbd86-71a1-4f33-a7a3-998d7267d4c7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417751', 'lastCheck': '6.7', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:09:57,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`d69fbd86-71a1-4f33-a7a3-998d7267d4c7`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:57,013::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:57,014::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:57,014::task::993::Storage.TaskManager.Task::(_decref) Task=`d69fbd86-71a1-4f33-a7a3-998d7267d4c7`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:09:57,034::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:09:58,069::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:09:58,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8bc9416-a267-46e1-8853-822ef1adac19`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:09:58,070::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:09:58,083::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:58,084::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8bc9416-a267-46e1-8853-822ef1adac19`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:09:58,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8bc9416-a267-46e1-8853-822ef1adac19`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:09:58,084::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:09:58,084::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:09:58,084::task::993::Storage.TaskManager.Task::(_decref) Task=`b8bc9416-a267-46e1-8853-822ef1adac19`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:09:58,095::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:09:58,095::task::595::Storage.TaskManager.Task::(_updateState) Task=`0adad40c-620e-499e-8b1c-d01e4f8ddcd3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:09:58,095::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:09:58,096::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d891e4c-4d41-4837-bc87-2ac59fdfdfaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:09:58,096::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:09:58,097::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:09:58,097::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d891e4c-4d41-4837-bc87-2ac59fdfdfaf`::Granted request >Thread-34::DEBUG::2014-08-11 15:09:58,098::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0adad40c-620e-499e-8b1c-d01e4f8ddcd3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:09:58,098::task::993::Storage.TaskManager.Task::(_decref) Task=`0adad40c-620e-499e-8b1c-d01e4f8ddcd3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:09:58,118::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:58,118::task::1191::Storage.TaskManager.Task::(prepare) Task=`0adad40c-620e-499e-8b1c-d01e4f8ddcd3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:09:58,118::task::595::Storage.TaskManager.Task::(_updateState) Task=`0adad40c-620e-499e-8b1c-d01e4f8ddcd3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:09:58,118::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:09:58,119::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:09:58,119::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:09:58,119::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:09:58,120::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:09:58,120::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:09:58,120::task::993::Storage.TaskManager.Task::(_decref) Task=`0adad40c-620e-499e-8b1c-d01e4f8ddcd3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:09:58,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:09:58,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132515 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:00,352::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:00,353::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:00,353::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:00,353::__init__::232::IOProcess::(_processLogs) (6455) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:00,353::__init__::232::IOProcess::(_processLogs) DEBUG|(6455) Queuing response >Thread-16::DEBUG::2014-08-11 15:10:00,354::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:10:00,373::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418265 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:00,374::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:00,375::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:00,375::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:00,375::__init__::232::IOProcess::(_processLogs) (6456) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:00,376::__init__::232::IOProcess::(_processLogs) DEBUG|(6456) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:00,377::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:00,377::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:00,377::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:00,378::__init__::232::IOProcess::(_processLogs) (6457) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:00,378::__init__::232::IOProcess::(_processLogs) (6457) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:00,379::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:00,379::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:00,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:00,379::__init__::232::IOProcess::(_processLogs) (6458) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:00,380::__init__::232::IOProcess::(_processLogs) (6458) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:00,380::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:00,381::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:00,381::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:00,381::__init__::232::IOProcess::(_processLogs) (6459) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:00,381::__init__::232::IOProcess::(_processLogs) (6459) Queuing response >Dummy-33::DEBUG::2014-08-11 15:10:00,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:00,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013541 s, 75.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:02,022::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:02,027::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:02,031::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:10:02,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:02,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0289867 s, 35.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:04,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:04,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132119 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:07,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:07,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131887 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:07,042::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:07,047::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:07,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:10:08,134::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:10:08,135::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bf13d21-dfa0-4abe-9d8c-66d9f1b4e98d`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:10:08,135::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:10:08,148::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:10:08,148::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bf13d21-dfa0-4abe-9d8c-66d9f1b4e98d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:10:08,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bf13d21-dfa0-4abe-9d8c-66d9f1b4e98d`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:10:08,149::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:10:08,149::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:10:08,149::task::993::Storage.TaskManager.Task::(_decref) Task=`8bf13d21-dfa0-4abe-9d8c-66d9f1b4e98d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:10:08,164::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:10:08,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb85555a-4f35-48d6-8d69-5cd7c2445c94`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:10:08,165::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:10:08,165::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3289a543-8e00-47ab-94de-4cec29d1685d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:10:08,166::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:10:08,166::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:10:08,166::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3289a543-8e00-47ab-94de-4cec29d1685d`::Granted request >Thread-34::DEBUG::2014-08-11 15:10:08,167::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bb85555a-4f35-48d6-8d69-5cd7c2445c94`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:10:08,167::task::993::Storage.TaskManager.Task::(_decref) Task=`bb85555a-4f35-48d6-8d69-5cd7c2445c94`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:10:08,187::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:08,187::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb85555a-4f35-48d6-8d69-5cd7c2445c94`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:08,187::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb85555a-4f35-48d6-8d69-5cd7c2445c94`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:10:08,188::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:10:08,188::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:10:08,188::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:10:08,189::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:10:08,189::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:10:08,189::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:10:08,189::task::993::Storage.TaskManager.Task::(_decref) Task=`bb85555a-4f35-48d6-8d69-5cd7c2445c94`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:10:09,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:09,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131663 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:10,383::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:10,384::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:10,384::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:10,384::__init__::232::IOProcess::(_processLogs) (6460) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:10,385::__init__::232::IOProcess::(_processLogs) DEBUG|(6460) Queuing response >Thread-16::DEBUG::2014-08-11 15:10:10,385::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:10:10,404::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428379 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:10,406::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:10,406::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:10,406::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:10,406::__init__::232::IOProcess::(_processLogs) (6461) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:10,407::__init__::232::IOProcess::(_processLogs) DEBUG|(6461) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:10,408::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:10,408::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:10,409::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:10,409::__init__::232::IOProcess::(_processLogs) DEBUG|(6462) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:10,409::__init__::232::IOProcess::(_processLogs) (6462) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:10,410::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:10,410::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:10,410::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:10,411::__init__::232::IOProcess::(_processLogs) DEBUG|(6463) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:10,411::__init__::232::IOProcess::(_processLogs) (6463) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:10,412::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:10,412::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:10,412::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:10,412::__init__::232::IOProcess::(_processLogs) DEBUG|(6464) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:10,413::__init__::232::IOProcess::(_processLogs) (6464) Queuing response >Dummy-33::DEBUG::2014-08-11 15:10:11,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:11,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131872 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:12,061::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:12,062::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:12,108::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-111::DEBUG::2014-08-11 15:10:12,129::task::595::Storage.TaskManager.Task::(_updateState) Task=`8de337ef-41b0-4703-97f7-db2479909b40`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:10:12,129::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-111::INFO::2014-08-11 15:10:12,130::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428379', 'lastCheck': '1.7', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:10:12,130::task::1191::Storage.TaskManager.Task::(prepare) Task=`8de337ef-41b0-4703-97f7-db2479909b40`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428379', 'lastCheck': '1.7', 'valid': True}} >Thread-111::DEBUG::2014-08-11 15:10:12,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`8de337ef-41b0-4703-97f7-db2479909b40`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:10:12,130::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:10:12,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:10:12,131::task::993::Storage.TaskManager.Task::(_decref) Task=`8de337ef-41b0-4703-97f7-db2479909b40`::ref 0 aborting False >Thread-111::DEBUG::2014-08-11 15:10:12,153::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:10:13,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:13,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131771 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:15,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:15,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132505 s, 77.3 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:10:16,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`59eb03c0-9a1c-441e-8327-1d5fb7443936`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:10:16,002::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:10:16,003::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:16,004::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:16,004::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:16,004::__init__::232::IOProcess::(_processLogs) (6465) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:10:16,005::__init__::232::IOProcess::(_processLogs) DEBUG|(6465) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:16,006::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-55::INFO::2014-08-11 15:10:16,007::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:10:16,007::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:16,008::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:10:16,008::task::1191::Storage.TaskManager.Task::(prepare) Task=`59eb03c0-9a1c-441e-8327-1d5fb7443936`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:10:16,009::__init__::232::IOProcess::(_processLogs) (6466) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:10:16,010::task::595::Storage.TaskManager.Task::(_updateState) Task=`59eb03c0-9a1c-441e-8327-1d5fb7443936`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:10:16,011::__init__::232::IOProcess::(_processLogs) (6466) Queuing response >Thread-55::DEBUG::2014-08-11 15:10:16,011::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:10:16,012::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:10:16,012::task::993::Storage.TaskManager.Task::(_decref) Task=`59eb03c0-9a1c-441e-8327-1d5fb7443936`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:17,115::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:17,116::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:17,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:10:17,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:17,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0285977 s, 35.8 MB/s\n'; <rc> = 0 >Thread-111::DEBUG::2014-08-11 15:10:18,203::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-111::DEBUG::2014-08-11 15:10:18,204::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bf79227-2b3d-41ef-9eed-ae4c0913a241`::moving from state init -> state preparing >Thread-111::INFO::2014-08-11 15:10:18,205::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-111::INFO::2014-08-11 15:10:18,218::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:10:18,219::task::1191::Storage.TaskManager.Task::(prepare) Task=`6bf79227-2b3d-41ef-9eed-ae4c0913a241`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-111::DEBUG::2014-08-11 15:10:18,219::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bf79227-2b3d-41ef-9eed-ae4c0913a241`::moving from state preparing -> state finished >Thread-111::DEBUG::2014-08-11 15:10:18,219::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-111::DEBUG::2014-08-11 15:10:18,219::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-111::DEBUG::2014-08-11 15:10:18,219::task::993::Storage.TaskManager.Task::(_decref) Task=`6bf79227-2b3d-41ef-9eed-ae4c0913a241`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:10:18,230::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:10:18,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5311c18-11c2-4e7d-af33-4685b7582826`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:10:18,231::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:10:18,232::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c17b3e0f-14bd-48b5-b844-ec2b4ad3af8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:10:18,232::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:10:18,232::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:10:18,233::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c17b3e0f-14bd-48b5-b844-ec2b4ad3af8f`::Granted request >Thread-34::DEBUG::2014-08-11 15:10:18,233::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a5311c18-11c2-4e7d-af33-4685b7582826`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:10:18,233::task::993::Storage.TaskManager.Task::(_decref) Task=`a5311c18-11c2-4e7d-af33-4685b7582826`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:10:18,253::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:18,254::task::1191::Storage.TaskManager.Task::(prepare) Task=`a5311c18-11c2-4e7d-af33-4685b7582826`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:18,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5311c18-11c2-4e7d-af33-4685b7582826`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:10:18,254::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:10:18,254::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:10:18,255::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:10:18,255::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:10:18,255::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:10:18,255::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:10:18,256::task::993::Storage.TaskManager.Task::(_decref) Task=`a5311c18-11c2-4e7d-af33-4685b7582826`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 15:10:18,658::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:42909 >Detector thread::DEBUG::2014-08-11 15:10:18,661::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:42909 >Detector thread::DEBUG::2014-08-11 15:10:18,662::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:42909 >Detector thread::DEBUG::2014-08-11 15:10:18,662::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 42909) >Dummy-33::DEBUG::2014-08-11 15:10:19,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:10:20,415::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:20,415::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:20,415::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:20,416::__init__::232::IOProcess::(_processLogs) (6467) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:20,886::__init__::232::IOProcess::(_processLogs) DEBUG|(6467) Queuing response >Thread-16::DEBUG::2014-08-11 15:10:20,889::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:20,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.32279 s, 774 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:10:20,915::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0006401 s, 711 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:20,918::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:20,919::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:20,921::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:20,922::__init__::232::IOProcess::(_processLogs) (6468) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:20,923::__init__::232::IOProcess::(_processLogs) (6468) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:20,923::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:20,924::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:20,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:20,926::__init__::232::IOProcess::(_processLogs) (6469) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:20,926::__init__::232::IOProcess::(_processLogs) (6469) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:20,927::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:20,928::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:20,929::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:20,930::__init__::232::IOProcess::(_processLogs) (6470) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:20,931::__init__::232::IOProcess::(_processLogs) (6470) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:20,933::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:20,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:20,935::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:20,937::__init__::232::IOProcess::(_processLogs) (6471) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:20,938::__init__::232::IOProcess::(_processLogs) (6471) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:22,216::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:22,217::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:22,309::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:10:22,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:22,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132683 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:25,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:25,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133232 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:27,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:27,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132212 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:27,316::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:27,318::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:27,409::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:10:27,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e250db5-f64d-42b6-a11e-7a41282e839b`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:10:27,702::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:10:27,703::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0006401', 'lastCheck': '6.8', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:10:27,703::task::1191::Storage.TaskManager.Task::(prepare) Task=`1e250db5-f64d-42b6-a11e-7a41282e839b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0006401', 'lastCheck': '6.8', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:10:27,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e250db5-f64d-42b6-a11e-7a41282e839b`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:10:27,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:10:27,704::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:10:27,704::task::993::Storage.TaskManager.Task::(_decref) Task=`1e250db5-f64d-42b6-a11e-7a41282e839b`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:10:27,723::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:10:28,267::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:10:28,268::task::595::Storage.TaskManager.Task::(_updateState) Task=`f42748da-4d4f-43e2-b938-1bfc18aa4304`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:10:28,268::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:10:28,281::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:28,281::task::1191::Storage.TaskManager.Task::(prepare) Task=`f42748da-4d4f-43e2-b938-1bfc18aa4304`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:28,282::task::595::Storage.TaskManager.Task::(_updateState) Task=`f42748da-4d4f-43e2-b938-1bfc18aa4304`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:10:28,282::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:10:28,282::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:10:28,282::task::993::Storage.TaskManager.Task::(_decref) Task=`f42748da-4d4f-43e2-b938-1bfc18aa4304`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:10:28,292::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:10:28,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`13b5edeb-3558-497e-9823-dc96c84f6dfb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:10:28,293::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:10:28,294::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0dd89bfa-dbfb-4fc6-a264-5273b0ddb281`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:10:28,294::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:10:28,295::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:10:28,295::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0dd89bfa-dbfb-4fc6-a264-5273b0ddb281`::Granted request >Thread-34::DEBUG::2014-08-11 15:10:28,295::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`13b5edeb-3558-497e-9823-dc96c84f6dfb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:10:28,296::task::993::Storage.TaskManager.Task::(_decref) Task=`13b5edeb-3558-497e-9823-dc96c84f6dfb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:10:28,315::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:28,316::task::1191::Storage.TaskManager.Task::(prepare) Task=`13b5edeb-3558-497e-9823-dc96c84f6dfb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136113152', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:28,316::task::595::Storage.TaskManager.Task::(_updateState) Task=`13b5edeb-3558-497e-9823-dc96c84f6dfb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:10:28,316::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:10:28,317::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:10:28,317::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:10:28,317::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:10:28,317::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:10:28,318::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:10:28,318::task::993::Storage.TaskManager.Task::(_decref) Task=`13b5edeb-3558-497e-9823-dc96c84f6dfb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:10:29,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:29,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013323 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:30,939::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:30,939::__init__::232::IOProcess::(_processLogs) (6472) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:10:30,940::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:10:30,959::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041603 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:30,961::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:30,961::__init__::232::IOProcess::(_processLogs) (6473) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:30,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:30,963::__init__::232::IOProcess::(_processLogs) (6474) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:30,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:30,964::__init__::232::IOProcess::(_processLogs) (6475) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:30,965::__init__::232::IOProcess::(_processLogs) DEBUG|(6475) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:30,966::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:30,966::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:30,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:30,968::__init__::232::IOProcess::(_processLogs) (6476) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:30,968::__init__::232::IOProcess::(_processLogs) (6476) Queuing response >Dummy-33::DEBUG::2014-08-11 15:10:31,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:31,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135586 s, 75.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:32,417::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:32,422::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:32,513::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:10:33,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:33,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013288 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:35,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:35,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134729 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:37,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:37,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132265 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:37,523::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:37,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:37,610::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:10:38,329::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:10:38,330::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c816799-c36b-4cf1-b52e-fbd7ab8d9150`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:10:38,330::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:10:38,343::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:38,343::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c816799-c36b-4cf1-b52e-fbd7ab8d9150`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:38,344::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c816799-c36b-4cf1-b52e-fbd7ab8d9150`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:10:38,344::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:10:38,344::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:10:38,344::task::993::Storage.TaskManager.Task::(_decref) Task=`4c816799-c36b-4cf1-b52e-fbd7ab8d9150`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:10:38,355::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:10:38,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`b38bfcce-4461-4eb6-ac15-d0775a767834`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:10:38,356::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:10:38,357::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fe60d1bd-f340-4f23-b789-cc33e5314e5b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:10:38,357::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:10:38,357::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:10:38,358::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fe60d1bd-f340-4f23-b789-cc33e5314e5b`::Granted request >Thread-34::DEBUG::2014-08-11 15:10:38,359::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b38bfcce-4461-4eb6-ac15-d0775a767834`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:10:38,359::task::993::Storage.TaskManager.Task::(_decref) Task=`b38bfcce-4461-4eb6-ac15-d0775a767834`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:10:38,379::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:38,380::task::1191::Storage.TaskManager.Task::(prepare) Task=`b38bfcce-4461-4eb6-ac15-d0775a767834`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:38,380::task::595::Storage.TaskManager.Task::(_updateState) Task=`b38bfcce-4461-4eb6-ac15-d0775a767834`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:10:38,380::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:10:38,380::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:10:38,381::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:10:38,381::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:10:38,381::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:10:38,381::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:10:38,382::task::993::Storage.TaskManager.Task::(_decref) Task=`b38bfcce-4461-4eb6-ac15-d0775a767834`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:10:39,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:39,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132487 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:40,969::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:40,970::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:10:40,971::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:10:40,972::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:40,989::__init__::232::IOProcess::(_processLogs) (6477) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:40,990::__init__::232::IOProcess::(_processLogs) (6477) Queuing response >Thread-16::DEBUG::2014-08-11 15:10:40,991::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000527895 s, 862 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:40,992::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:40,992::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:40,993::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:40,993::__init__::232::IOProcess::(_processLogs) (6478) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:40,994::__init__::232::IOProcess::(_processLogs) DEBUG|(6478) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:40,995::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:40,996::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:40,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:40,997::__init__::232::IOProcess::(_processLogs) (6479) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:40,997::__init__::232::IOProcess::(_processLogs) (6479) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:40,997::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:40,998::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:40,999::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:40,999::__init__::232::IOProcess::(_processLogs) (6480) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:40,999::__init__::232::IOProcess::(_processLogs) (6480) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:40,999::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:41,000::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:41,001::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:41,001::__init__::232::IOProcess::(_processLogs) (6481) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:41,001::__init__::232::IOProcess::(_processLogs) (6481) Queuing response >Dummy-33::DEBUG::2014-08-11 15:10:41,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:41,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132259 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:42,617::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:42,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:42,710::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:10:42,824::task::595::Storage.TaskManager.Task::(_updateState) Task=`904473d5-ea3d-43ef-8d2b-9857611b8060`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:10:42,824::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:10:42,825::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000527895', 'lastCheck': '1.8', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:10:42,825::task::1191::Storage.TaskManager.Task::(prepare) Task=`904473d5-ea3d-43ef-8d2b-9857611b8060`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000527895', 'lastCheck': '1.8', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:10:42,825::task::595::Storage.TaskManager.Task::(_updateState) Task=`904473d5-ea3d-43ef-8d2b-9857611b8060`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:10:42,826::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:10:42,826::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:10:42,826::task::993::Storage.TaskManager.Task::(_decref) Task=`904473d5-ea3d-43ef-8d2b-9857611b8060`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:10:42,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:10:43,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:43,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137765 s, 74.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:45,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:45,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132324 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:47,621::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:47,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133289 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:47,718::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:47,723::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:47,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:10:48,398::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:10:48,399::task::595::Storage.TaskManager.Task::(_updateState) Task=`571ed709-c147-4a6e-a79a-32cc423e9bb1`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:10:48,400::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:10:48,413::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:48,413::task::1191::Storage.TaskManager.Task::(prepare) Task=`571ed709-c147-4a6e-a79a-32cc423e9bb1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:48,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`571ed709-c147-4a6e-a79a-32cc423e9bb1`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:10:48,414::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:10:48,414::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:10:48,414::task::993::Storage.TaskManager.Task::(_decref) Task=`571ed709-c147-4a6e-a79a-32cc423e9bb1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:10:48,425::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:10:48,425::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c9cd5e6-db7e-48b7-a484-4b22cf79f798`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:10:48,426::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:10:48,426::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b181f5f4-e8da-4c0a-b402-b5c696197509`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:10:48,427::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:10:48,427::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:10:48,427::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b181f5f4-e8da-4c0a-b402-b5c696197509`::Granted request >Thread-34::DEBUG::2014-08-11 15:10:48,428::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8c9cd5e6-db7e-48b7-a484-4b22cf79f798`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:10:48,428::task::993::Storage.TaskManager.Task::(_decref) Task=`8c9cd5e6-db7e-48b7-a484-4b22cf79f798`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:10:48,448::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:48,448::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c9cd5e6-db7e-48b7-a484-4b22cf79f798`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:48,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c9cd5e6-db7e-48b7-a484-4b22cf79f798`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:10:48,449::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:10:48,449::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:10:48,449::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:10:48,450::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:10:48,450::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:10:48,450::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:10:48,450::task::993::Storage.TaskManager.Task::(_decref) Task=`8c9cd5e6-db7e-48b7-a484-4b22cf79f798`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:10:49,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:49,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132735 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:51,002::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:51,003::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:10:51,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:10:51,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:51,023::__init__::232::IOProcess::(_processLogs) (6482) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:51,024::__init__::232::IOProcess::(_processLogs) (6482) Queuing response >Thread-16::DEBUG::2014-08-11 15:10:51,024::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000476731 s, 954 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:10:51,025::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:51,026::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:51,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:51,026::__init__::232::IOProcess::(_processLogs) (6483) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:10:51,027::__init__::232::IOProcess::(_processLogs) DEBUG|(6483) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:51,028::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:51,029::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:51,030::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:51,030::__init__::232::IOProcess::(_processLogs) (6484) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:51,030::__init__::232::IOProcess::(_processLogs) (6484) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:51,030::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:51,031::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:51,032::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:51,032::__init__::232::IOProcess::(_processLogs) (6485) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:51,032::__init__::232::IOProcess::(_processLogs) (6485) Queuing response >Thread-14::DEBUG::2014-08-11 15:10:51,033::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:10:51,033::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:10:51,034::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:10:51,035::__init__::232::IOProcess::(_processLogs) (6486) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:10:51,035::__init__::232::IOProcess::(_processLogs) (6486) Queuing response >Dummy-33::DEBUG::2014-08-11 15:10:51,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:51,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132554 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:52,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:52,738::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:52,811::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:10:53,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:53,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131716 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:10:55,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:57,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.46206 s, 700 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:57,814::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:57,816::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:10:57,817::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:10:57,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`f100acba-f042-47fa-b08f-d099e3b682c9`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:10:57,997::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:10:57,997::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000476731', 'lastCheck': '7.0', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:10:57,997::task::1191::Storage.TaskManager.Task::(prepare) Task=`f100acba-f042-47fa-b08f-d099e3b682c9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000476731', 'lastCheck': '7.0', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:10:57,997::task::595::Storage.TaskManager.Task::(_updateState) Task=`f100acba-f042-47fa-b08f-d099e3b682c9`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:10:57,998::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:10:57,998::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:10:57,998::task::993::Storage.TaskManager.Task::(_decref) Task=`f100acba-f042-47fa-b08f-d099e3b682c9`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:10:58,019::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:10:58,463::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:10:58,463::task::595::Storage.TaskManager.Task::(_updateState) Task=`c81dd99c-21a9-4868-9e17-978f8db292fe`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:10:58,464::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:10:58,476::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:58,477::task::1191::Storage.TaskManager.Task::(prepare) Task=`c81dd99c-21a9-4868-9e17-978f8db292fe`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:10:58,477::task::595::Storage.TaskManager.Task::(_updateState) Task=`c81dd99c-21a9-4868-9e17-978f8db292fe`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:10:58,477::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:10:58,478::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:10:58,478::task::993::Storage.TaskManager.Task::(_decref) Task=`c81dd99c-21a9-4868-9e17-978f8db292fe`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:10:58,490::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:10:58,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3780455-58c1-48ae-8c0b-f6f4dc08d430`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:10:58,491::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:10:58,492::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b31e9e58-caa9-4609-938f-3edb03db9fe0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:10:58,492::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:10:58,492::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:10:58,493::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b31e9e58-caa9-4609-938f-3edb03db9fe0`::Granted request >Thread-34::DEBUG::2014-08-11 15:10:58,493::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f3780455-58c1-48ae-8c0b-f6f4dc08d430`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:10:58,494::task::993::Storage.TaskManager.Task::(_decref) Task=`f3780455-58c1-48ae-8c0b-f6f4dc08d430`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:10:58,513::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:58,514::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3780455-58c1-48ae-8c0b-f6f4dc08d430`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:10:58,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3780455-58c1-48ae-8c0b-f6f4dc08d430`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:10:58,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:10:58,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:10:58,515::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:10:58,515::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:10:58,515::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:10:58,515::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:10:58,516::task::993::Storage.TaskManager.Task::(_decref) Task=`f3780455-58c1-48ae-8c0b-f6f4dc08d430`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:10:59,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:10:59,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013226 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:01,036::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:01,037::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:11:01,038::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:11:01,039::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:01,056::__init__::232::IOProcess::(_processLogs) (6487) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:01,057::__init__::232::IOProcess::(_processLogs) (6487) Queuing response >Thread-16::DEBUG::2014-08-11 15:11:01,058::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000593444 s, 767 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:01,061::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:01,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:01,064::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:01,064::__init__::232::IOProcess::(_processLogs) (6488) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:01,065::__init__::232::IOProcess::(_processLogs) DEBUG|(6488) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:01,066::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:01,066::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:01,067::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:01,067::__init__::232::IOProcess::(_processLogs) (6489) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:01,067::__init__::232::IOProcess::(_processLogs) DEBUG|(6489) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:01,068::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:01,069::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:01,069::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:01,070::__init__::232::IOProcess::(_processLogs) (6490) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:01,070::__init__::232::IOProcess::(_processLogs) (6490) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:01,071::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:01,071::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:01,072::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:01,073::__init__::232::IOProcess::(_processLogs) (6491) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:01,073::__init__::232::IOProcess::(_processLogs) (6491) Queuing response >Dummy-33::DEBUG::2014-08-11 15:11:01,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:01,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135778 s, 75.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:02,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:02,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:02,916::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:11:03,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:03,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133232 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:05,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:05,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131921 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:07,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:07,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132537 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:07,929::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:07,934::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:08,011::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:08,527::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:11:08,528::task::595::Storage.TaskManager.Task::(_updateState) Task=`982f28cf-9bfd-4a30-be27-16d02d82447c`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:08,529::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:11:08,542::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:08,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`982f28cf-9bfd-4a30-be27-16d02d82447c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:08,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`982f28cf-9bfd-4a30-be27-16d02d82447c`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:08,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:08,543::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:08,543::task::993::Storage.TaskManager.Task::(_decref) Task=`982f28cf-9bfd-4a30-be27-16d02d82447c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:11:08,554::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:11:08,555::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d653caf-ec59-4e2d-a9b4-5cf5c684ed7b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:11:08,555::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:11:08,556::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5ae6c17b-7d15-44ab-a940-6c4a69115da5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:11:08,556::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:11:08,556::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:11:08,556::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5ae6c17b-7d15-44ab-a940-6c4a69115da5`::Granted request >Thread-34::DEBUG::2014-08-11 15:11:08,557::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1d653caf-ec59-4e2d-a9b4-5cf5c684ed7b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:11:08,557::task::993::Storage.TaskManager.Task::(_decref) Task=`1d653caf-ec59-4e2d-a9b4-5cf5c684ed7b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:11:08,577::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:08,577::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d653caf-ec59-4e2d-a9b4-5cf5c684ed7b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:08,578::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d653caf-ec59-4e2d-a9b4-5cf5c684ed7b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:11:08,578::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:11:08,578::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:11:08,578::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:11:08,579::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:11:08,579::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:11:08,579::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:11:08,579::task::993::Storage.TaskManager.Task::(_decref) Task=`1d653caf-ec59-4e2d-a9b4-5cf5c684ed7b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:11:09,689::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:09,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132979 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:11,074::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:11,075::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:11,076::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:11:11,077::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:11:11,078::__init__::232::IOProcess::(_processLogs) (6492) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:11,096::__init__::232::IOProcess::(_processLogs) (6492) Queuing response >Thread-16::DEBUG::2014-08-11 15:11:11,097::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000543205 s, 838 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:11,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:11,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:11,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:11,099::__init__::232::IOProcess::(_processLogs) (6493) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:11,100::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6493) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:11,101::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:11,101::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:11,102::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:11,102::__init__::232::IOProcess::(_processLogs) (6494) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:11,102::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6494) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:11,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:11,104::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:11,104::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:11,104::__init__::232::IOProcess::(_processLogs) (6495) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:11,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6495) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:11,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:11,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:11,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:11,106::__init__::232::IOProcess::(_processLogs) (6496) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:11,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6496) Queuing response >Dummy-33::DEBUG::2014-08-11 15:11:11,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:11,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132226 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:13,015::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:13,017::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:13,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:13,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`fee5c1ee-faf0-43a4-b855-9f709850cb7f`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:13,117::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:11:13,117::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000543205', 'lastCheck': '2.0', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:11:13,117::task::1191::Storage.TaskManager.Task::(prepare) Task=`fee5c1ee-faf0-43a4-b855-9f709850cb7f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000543205', 'lastCheck': '2.0', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:11:13,118::task::595::Storage.TaskManager.Task::(_updateState) Task=`fee5c1ee-faf0-43a4-b855-9f709850cb7f`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:13,118::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:13,118::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:13,118::task::993::Storage.TaskManager.Task::(_decref) Task=`fee5c1ee-faf0-43a4-b855-9f709850cb7f`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:11:13,140::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:11:13,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:13,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013216 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:15,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:15,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013297 s, 77.0 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:11:16,119::task::595::Storage.TaskManager.Task::(_updateState) Task=`96b33bf9-4ec1-42e1-aebc-814714560a1a`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:11:16,119::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:11:16,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:16,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:16,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:16,121::__init__::232::IOProcess::(_processLogs) (6497) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:11:16,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6497) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:16,123::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:16,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:16,124::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:16,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6498) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:11:16,124::__init__::232::IOProcess::(_processLogs) (6498) Queuing response >Thread-55::INFO::2014-08-11 15:11:16,125::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:11:16,125::task::1191::Storage.TaskManager.Task::(prepare) Task=`96b33bf9-4ec1-42e1-aebc-814714560a1a`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:11:16,125::task::595::Storage.TaskManager.Task::(_updateState) Task=`96b33bf9-4ec1-42e1-aebc-814714560a1a`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:11:16,126::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:11:16,126::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:11:16,126::task::993::Storage.TaskManager.Task::(_decref) Task=`96b33bf9-4ec1-42e1-aebc-814714560a1a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:11:17,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:17,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132412 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:18,117::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:18,118::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:18,119::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:18,593::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:11:18,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa4c3aaa-cf57-43c9-b706-89dcde91ec7a`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:18,594::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:11:18,607::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:18,608::task::1191::Storage.TaskManager.Task::(prepare) Task=`aa4c3aaa-cf57-43c9-b706-89dcde91ec7a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:18,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`aa4c3aaa-cf57-43c9-b706-89dcde91ec7a`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:18,608::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:18,608::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:18,608::task::993::Storage.TaskManager.Task::(_decref) Task=`aa4c3aaa-cf57-43c9-b706-89dcde91ec7a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:11:18,618::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:11:18,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`331c058f-2ed5-4603-aaa4-9cc06754bda0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:11:18,620::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:11:18,620::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4131191b-5c42-4597-9881-8aaff8cf4969`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:11:18,621::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:11:18,621::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:11:18,621::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4131191b-5c42-4597-9881-8aaff8cf4969`::Granted request >Thread-34::DEBUG::2014-08-11 15:11:18,622::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`331c058f-2ed5-4603-aaa4-9cc06754bda0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:11:18,622::task::993::Storage.TaskManager.Task::(_decref) Task=`331c058f-2ed5-4603-aaa4-9cc06754bda0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:11:18,642::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:18,642::task::1191::Storage.TaskManager.Task::(prepare) Task=`331c058f-2ed5-4603-aaa4-9cc06754bda0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:18,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`331c058f-2ed5-4603-aaa4-9cc06754bda0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:11:18,643::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:11:18,643::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:11:18,643::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:11:18,643::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:11:18,644::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:11:18,644::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:11:18,644::task::993::Storage.TaskManager.Task::(_decref) Task=`331c058f-2ed5-4603-aaa4-9cc06754bda0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:11:19,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:19,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132447 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:21,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:21,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:21,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:21,110::__init__::232::IOProcess::(_processLogs) (6499) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:21,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6499) Queuing response >Thread-16::DEBUG::2014-08-11 15:11:21,111::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:11:21,130::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422202 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:21,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:21,132::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:21,132::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:21,132::__init__::232::IOProcess::(_processLogs) (6500) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:21,132::__init__::232::IOProcess::(_processLogs) (6500) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:21,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:21,134::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:21,134::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:21,134::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6501) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:21,135::__init__::232::IOProcess::(_processLogs) (6501) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:21,135::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:21,136::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:21,136::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:21,136::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6502) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:21,136::__init__::232::IOProcess::(_processLogs) (6502) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:21,137::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:21,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:21,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:21,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6503) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:21,138::__init__::232::IOProcess::(_processLogs) (6503) Queuing response >Dummy-33::DEBUG::2014-08-11 15:11:22,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:22,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132198 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:23,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:23,127::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:23,211::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:11:24,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:24,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132479 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:26,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:26,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133352 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:28,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:28,219::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:28,222::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:28,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:11:28,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0149516 s, 68.5 MB/s\n'; <rc> = 0 >Thread-113::DEBUG::2014-08-11 15:11:28,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7ae281c-ff29-4022-8838-14d7e4f1d304`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:28,246::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:11:28,248::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422202', 'lastCheck': '7.1', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:11:28,249::task::1191::Storage.TaskManager.Task::(prepare) Task=`a7ae281c-ff29-4022-8838-14d7e4f1d304`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422202', 'lastCheck': '7.1', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:11:28,250::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7ae281c-ff29-4022-8838-14d7e4f1d304`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:28,251::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:28,252::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:28,253::task::993::Storage.TaskManager.Task::(_decref) Task=`a7ae281c-ff29-4022-8838-14d7e4f1d304`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:11:28,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:28,656::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:11:28,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd0a7b5d-f7ac-4f99-9d1e-7a6162154465`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:28,657::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:11:28,669::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:28,670::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd0a7b5d-f7ac-4f99-9d1e-7a6162154465`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:28,670::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd0a7b5d-f7ac-4f99-9d1e-7a6162154465`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:28,670::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:28,671::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:28,671::task::993::Storage.TaskManager.Task::(_decref) Task=`dd0a7b5d-f7ac-4f99-9d1e-7a6162154465`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:11:28,681::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:11:28,681::task::595::Storage.TaskManager.Task::(_updateState) Task=`83616f33-a835-43a7-8726-819b75daf80e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:11:28,682::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:11:28,682::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7c96f935-dd0d-4bae-846f-042f78a641ab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:11:28,683::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:11:28,683::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:11:28,683::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7c96f935-dd0d-4bae-846f-042f78a641ab`::Granted request >Thread-34::DEBUG::2014-08-11 15:11:28,684::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`83616f33-a835-43a7-8726-819b75daf80e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:11:28,684::task::993::Storage.TaskManager.Task::(_decref) Task=`83616f33-a835-43a7-8726-819b75daf80e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:11:28,704::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:28,704::task::1191::Storage.TaskManager.Task::(prepare) Task=`83616f33-a835-43a7-8726-819b75daf80e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:28,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`83616f33-a835-43a7-8726-819b75daf80e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:11:28,705::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:11:28,705::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:11:28,705::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:11:28,706::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:11:28,706::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:11:28,706::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:11:28,706::task::993::Storage.TaskManager.Task::(_decref) Task=`83616f33-a835-43a7-8726-819b75daf80e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:11:30,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:30,315::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134071 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:31,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:31,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:31,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:31,141::__init__::232::IOProcess::(_processLogs) (6504) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:31,142::__init__::232::IOProcess::(_processLogs) (6504) Queuing response >Thread-16::DEBUG::2014-08-11 15:11:31,142::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:11:31,161::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041663 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:31,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:31,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:31,163::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:31,163::__init__::232::IOProcess::(_processLogs) (6505) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:31,163::__init__::232::IOProcess::(_processLogs) (6505) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:31,164::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:31,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:31,166::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:31,166::__init__::232::IOProcess::(_processLogs) (6506) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:31,166::__init__::232::IOProcess::(_processLogs) (6506) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:31,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:31,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:31,167::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:31,167::__init__::232::IOProcess::(_processLogs) (6507) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:31,168::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6507) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:31,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:31,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:31,170::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:31,171::__init__::232::IOProcess::(_processLogs) (6508) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:31,171::__init__::232::IOProcess::(_processLogs) (6508) Queuing response >Dummy-33::DEBUG::2014-08-11 15:11:32,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:32,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278524 s, 36.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:33,232::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:33,234::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:33,311::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:11:34,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:34,650::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132252 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:36,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:36,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132135 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:38,318::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:38,319::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:38,320::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:38,718::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:11:38,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce01eecd-953a-412b-bc7f-266f09f804df`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:38,719::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 15:11:38,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-113::INFO::2014-08-11 15:11:38,746::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:38,747::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce01eecd-953a-412b-bc7f-266f09f804df`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:38,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce01eecd-953a-412b-bc7f-266f09f804df`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:38,747::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:38,748::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:38,748::task::993::Storage.TaskManager.Task::(_decref) Task=`ce01eecd-953a-412b-bc7f-266f09f804df`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:11:38,761::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:11:38,764::task::595::Storage.TaskManager.Task::(_updateState) Task=`2404b8a0-77a7-4917-b2e1-26e583ae7056`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:11:38,765::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 15:11:38,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0178282 s, 57.4 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 15:11:38,769::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`224ea73f-569b-4f35-a72a-bcb9ff9330f7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:11:38,770::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:11:38,772::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:11:38,773::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`224ea73f-569b-4f35-a72a-bcb9ff9330f7`::Granted request >Thread-34::DEBUG::2014-08-11 15:11:38,775::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2404b8a0-77a7-4917-b2e1-26e583ae7056`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:11:38,776::task::993::Storage.TaskManager.Task::(_decref) Task=`2404b8a0-77a7-4917-b2e1-26e583ae7056`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:11:38,826::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:38,826::task::1191::Storage.TaskManager.Task::(prepare) Task=`2404b8a0-77a7-4917-b2e1-26e583ae7056`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:38,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`2404b8a0-77a7-4917-b2e1-26e583ae7056`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:11:38,826::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:11:38,827::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:11:38,827::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:11:38,827::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:11:38,828::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:11:38,828::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:11:38,828::task::993::Storage.TaskManager.Task::(_decref) Task=`2404b8a0-77a7-4917-b2e1-26e583ae7056`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:11:40,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:40,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131562 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:41,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:41,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:41,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:41,175::__init__::232::IOProcess::(_processLogs) (6509) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:11:41,175::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:11:41,176::__init__::232::IOProcess::(_processLogs) (6509) Queuing response >Thread-16::DEBUG::2014-08-11 15:11:41,195::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420151 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:41,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:41,197::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:41,197::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:41,197::__init__::232::IOProcess::(_processLogs) (6510) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:41,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6510) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:41,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:41,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:41,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:41,201::__init__::232::IOProcess::(_processLogs) (6511) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:41,201::__init__::232::IOProcess::(_processLogs) (6511) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:41,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:41,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:41,203::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:41,203::__init__::232::IOProcess::(_processLogs) (6512) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:41,203::__init__::232::IOProcess::(_processLogs) (6512) Queuing response >Thread-14::DEBUG::2014-08-11 15:11:41,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:41,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:11:41,205::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:41,206::__init__::232::IOProcess::(_processLogs) (6513) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:41,206::__init__::232::IOProcess::(_processLogs) (6513) Queuing response >Dummy-33::DEBUG::2014-08-11 15:11:42,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:42,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133369 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:43,327::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:43,328::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:43,384::task::595::Storage.TaskManager.Task::(_updateState) Task=`807edba3-38d6-4842-9817-43d4638a35c1`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:43,412::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:11:43,414::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420151', 'lastCheck': '2.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:43,415::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:43,416::task::1191::Storage.TaskManager.Task::(prepare) Task=`807edba3-38d6-4842-9817-43d4638a35c1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420151', 'lastCheck': '2.2', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:11:43,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`807edba3-38d6-4842-9817-43d4638a35c1`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:43,418::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:43,418::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:43,419::task::993::Storage.TaskManager.Task::(_decref) Task=`807edba3-38d6-4842-9817-43d4638a35c1`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:11:43,438::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:11:44,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:44,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013235 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:46,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:47,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132243 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:48,424::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:48,425::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:48,426::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:48,841::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:11:48,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`73e6bd9e-ab99-4764-a910-019562a9d70b`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:48,842::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:11:48,854::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:48,855::task::1191::Storage.TaskManager.Task::(prepare) Task=`73e6bd9e-ab99-4764-a910-019562a9d70b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:48,855::task::595::Storage.TaskManager.Task::(_updateState) Task=`73e6bd9e-ab99-4764-a910-019562a9d70b`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:48,855::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:48,855::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:48,856::task::993::Storage.TaskManager.Task::(_decref) Task=`73e6bd9e-ab99-4764-a910-019562a9d70b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:11:48,866::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:11:48,866::task::595::Storage.TaskManager.Task::(_updateState) Task=`c73cbb11-e04d-4f1d-b1c7-d3916c996fa0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:11:48,866::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:11:48,867::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a0b1429-b1d6-41c3-88ef-c6003f142cd1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:11:48,867::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:11:48,868::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:11:48,868::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a0b1429-b1d6-41c3-88ef-c6003f142cd1`::Granted request >Thread-34::DEBUG::2014-08-11 15:11:48,869::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c73cbb11-e04d-4f1d-b1c7-d3916c996fa0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:11:48,869::task::993::Storage.TaskManager.Task::(_decref) Task=`c73cbb11-e04d-4f1d-b1c7-d3916c996fa0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:11:48,889::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:48,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`c73cbb11-e04d-4f1d-b1c7-d3916c996fa0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:48,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`c73cbb11-e04d-4f1d-b1c7-d3916c996fa0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:11:48,889::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:11:48,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:11:48,890::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:11:48,890::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:11:48,891::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:11:48,891::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:11:48,891::task::993::Storage.TaskManager.Task::(_decref) Task=`c73cbb11-e04d-4f1d-b1c7-d3916c996fa0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:11:49,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:49,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132696 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:51,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:51,142::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132845 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:51,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:11:51,207::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:11:51,208::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:11:51,209::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:51,227::__init__::232::IOProcess::(_processLogs) (6514) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:51,227::__init__::232::IOProcess::(_processLogs) (6514) Queuing response >Thread-16::DEBUG::2014-08-11 15:11:51,228::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419911 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:11:51,229::__init__::232::IOProcess::(_processLogs) (6515) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:11:51,231::__init__::232::IOProcess::(_processLogs) (6516) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:51,231::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:51,232::__init__::232::IOProcess::(_processLogs) (6517) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:51,233::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:11:51,233::__init__::232::IOProcess::(_processLogs) (6518) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:11:51,233::__init__::232::IOProcess::(_processLogs) (6518) Queuing response >Dummy-33::DEBUG::2014-08-11 15:11:53,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:53,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013153 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:53,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:53,434::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:53,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:11:55,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:55,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132054 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:11:57,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:57,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131465 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:58,515::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:58,516::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:11:58,517::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:58,541::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce20a02c-aae5-4f13-8b7d-c2177f171f74`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:58,541::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:11:58,542::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419911', 'lastCheck': '7.3', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:11:58,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce20a02c-aae5-4f13-8b7d-c2177f171f74`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419911', 'lastCheck': '7.3', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:11:58,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce20a02c-aae5-4f13-8b7d-c2177f171f74`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:58,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:58,543::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:58,543::task::993::Storage.TaskManager.Task::(_decref) Task=`ce20a02c-aae5-4f13-8b7d-c2177f171f74`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:11:58,563::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:11:58,903::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:11:58,904::task::595::Storage.TaskManager.Task::(_updateState) Task=`80ae221c-548b-4ba5-9fc2-0cc6b1388493`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:11:58,904::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:11:58,917::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:58,917::task::1191::Storage.TaskManager.Task::(prepare) Task=`80ae221c-548b-4ba5-9fc2-0cc6b1388493`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:11:58,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`80ae221c-548b-4ba5-9fc2-0cc6b1388493`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:11:58,917::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:11:58,918::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:11:58,918::task::993::Storage.TaskManager.Task::(_decref) Task=`80ae221c-548b-4ba5-9fc2-0cc6b1388493`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:11:58,928::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:11:58,929::task::595::Storage.TaskManager.Task::(_updateState) Task=`f828c241-f864-4960-bb7d-f459cc52ca29`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:11:58,929::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:11:58,930::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2e693c7f-9de8-4ccd-94ad-ac49bd385e4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:11:58,930::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:11:58,931::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:11:58,931::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2e693c7f-9de8-4ccd-94ad-ac49bd385e4a`::Granted request >Thread-34::DEBUG::2014-08-11 15:11:58,931::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f828c241-f864-4960-bb7d-f459cc52ca29`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:11:58,932::task::993::Storage.TaskManager.Task::(_decref) Task=`f828c241-f864-4960-bb7d-f459cc52ca29`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:11:58,952::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:58,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`f828c241-f864-4960-bb7d-f459cc52ca29`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:11:58,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`f828c241-f864-4960-bb7d-f459cc52ca29`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:11:58,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:11:58,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:11:58,953::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:11:58,953::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:11:58,954::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:11:58,954::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:11:58,954::task::993::Storage.TaskManager.Task::(_decref) Task=`f828c241-f864-4960-bb7d-f459cc52ca29`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:11:59,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:11:59,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132786 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:01,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:01,236::__init__::232::IOProcess::(_processLogs) (6519) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:12:01,237::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:12:01,277::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00805994 s, 56.5 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:01,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:01,280::__init__::232::IOProcess::(_processLogs) (6520) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:01,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:01,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:01,282::__init__::232::IOProcess::(_processLogs) (6521) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:01,282::__init__::232::IOProcess::(_processLogs) (6521) Queuing response >Thread-14::DEBUG::2014-08-11 15:12:01,283::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:01,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:01,284::__init__::232::IOProcess::(_processLogs) (6522) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:01,284::__init__::232::IOProcess::(_processLogs) (6522) Queuing response >Thread-14::DEBUG::2014-08-11 15:12:01,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:01,286::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:01,286::__init__::232::IOProcess::(_processLogs) (6523) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:01,286::__init__::232::IOProcess::(_processLogs) (6523) Queuing response >Dummy-33::DEBUG::2014-08-11 15:12:01,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:01,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134164 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:03,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:03,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131666 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:03,524::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:03,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:03,611::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:12:05,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:05,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132517 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:07,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:07,581::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131917 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:08,618::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:08,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:08,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:08,965::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:12:08,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b97c329-163e-47ed-942d-c01b823c05f8`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:08,967::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:12:08,980::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:08,980::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b97c329-163e-47ed-942d-c01b823c05f8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:08,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b97c329-163e-47ed-942d-c01b823c05f8`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:08,980::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:08,981::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:08,981::task::993::Storage.TaskManager.Task::(_decref) Task=`0b97c329-163e-47ed-942d-c01b823c05f8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:12:08,992::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:12:08,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`7867a02b-0e16-429b-84ca-0a6ef223fe5f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:12:08,993::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:12:08,994::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1555c670-3b97-40ec-a076-d7b601abba04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:12:08,994::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:12:08,994::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:12:08,994::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1555c670-3b97-40ec-a076-d7b601abba04`::Granted request >Thread-34::DEBUG::2014-08-11 15:12:08,995::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7867a02b-0e16-429b-84ca-0a6ef223fe5f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:12:08,995::task::993::Storage.TaskManager.Task::(_decref) Task=`7867a02b-0e16-429b-84ca-0a6ef223fe5f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:12:09,015::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:09,015::task::1191::Storage.TaskManager.Task::(prepare) Task=`7867a02b-0e16-429b-84ca-0a6ef223fe5f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:09,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`7867a02b-0e16-429b-84ca-0a6ef223fe5f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:12:09,016::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:12:09,016::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:12:09,016::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:12:09,017::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:12:09,017::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:12:09,017::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:12:09,017::task::993::Storage.TaskManager.Task::(_decref) Task=`7867a02b-0e16-429b-84ca-0a6ef223fe5f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:12:09,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:09,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132588 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:11,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:11,289::__init__::232::IOProcess::(_processLogs) (6524) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:12:11,290::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:12:11,291::__init__::232::IOProcess::(_processLogs) (6524) Queuing response >Thread-16::DEBUG::2014-08-11 15:12:11,309::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419581 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:11,311::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:11,311::__init__::232::IOProcess::(_processLogs) (6525) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:11,312::__init__::232::IOProcess::(_processLogs) (6526) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:11,313::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:11,314::__init__::232::IOProcess::(_processLogs) (6527) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:11,315::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:11,315::__init__::232::IOProcess::(_processLogs) (6528) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:11,315::__init__::232::IOProcess::(_processLogs) (6528) Queuing response >Dummy-33::DEBUG::2014-08-11 15:12:11,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:11,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132671 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:13,627::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:13,628::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:13,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3f75704-ca4a-4816-a4e4-fadce7144ea1`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:13,713::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Dummy-33::DEBUG::2014-08-11 15:12:13,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-113::INFO::2014-08-11 15:12:13,715::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419581', 'lastCheck': '2.4', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:13,735::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:13,735::task::1191::Storage.TaskManager.Task::(prepare) Task=`f3f75704-ca4a-4816-a4e4-fadce7144ea1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419581', 'lastCheck': '2.4', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:12:13,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`f3f75704-ca4a-4816-a4e4-fadce7144ea1`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:13,739::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:13,740::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:13,740::task::993::Storage.TaskManager.Task::(_decref) Task=`f3f75704-ca4a-4816-a4e4-fadce7144ea1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:12:13,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0154092 s, 66.5 MB/s\n'; <rc> = 0 >Thread-113::DEBUG::2014-08-11 15:12:13,764::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:12:15,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:15,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132349 s, 77.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:12:16,295::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e89057a-1572-463b-bd6d-d9976985203e`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:12:16,296::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:12:16,297::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:16,297::__init__::232::IOProcess::(_processLogs) (6529) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:12:16,299::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:16,299::__init__::232::IOProcess::(_processLogs) (6530) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:12:16,300::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:12:16,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e89057a-1572-463b-bd6d-d9976985203e`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:12:16,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e89057a-1572-463b-bd6d-d9976985203e`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:12:16,301::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:12:16,301::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:12:16,301::task::993::Storage.TaskManager.Task::(_decref) Task=`4e89057a-1572-463b-bd6d-d9976985203e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:12:17,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:17,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131223 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:18,745::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:18,746::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:18,747::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:19,030::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:12:19,031::task::595::Storage.TaskManager.Task::(_updateState) Task=`190f9ddc-5fe7-4dca-ae6f-e3f3399a800e`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:19,031::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:12:19,044::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:19,045::task::1191::Storage.TaskManager.Task::(prepare) Task=`190f9ddc-5fe7-4dca-ae6f-e3f3399a800e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:19,045::task::595::Storage.TaskManager.Task::(_updateState) Task=`190f9ddc-5fe7-4dca-ae6f-e3f3399a800e`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:19,045::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:19,045::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:19,046::task::993::Storage.TaskManager.Task::(_decref) Task=`190f9ddc-5fe7-4dca-ae6f-e3f3399a800e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:12:19,055::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:12:19,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`c51540af-6285-4150-933c-334fd392b809`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:12:19,056::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:12:19,057::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`408af477-f2c1-4350-8e29-df1d7dcffcdb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:12:19,057::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:12:19,058::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:12:19,058::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`408af477-f2c1-4350-8e29-df1d7dcffcdb`::Granted request >Thread-34::DEBUG::2014-08-11 15:12:19,058::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c51540af-6285-4150-933c-334fd392b809`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:12:19,059::task::993::Storage.TaskManager.Task::(_decref) Task=`c51540af-6285-4150-933c-334fd392b809`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:12:19,079::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:19,079::task::1191::Storage.TaskManager.Task::(prepare) Task=`c51540af-6285-4150-933c-334fd392b809`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:19,079::task::595::Storage.TaskManager.Task::(_updateState) Task=`c51540af-6285-4150-933c-334fd392b809`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:12:19,080::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:12:19,080::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:12:19,080::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:12:19,081::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:12:19,081::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:12:19,081::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:12:19,081::task::993::Storage.TaskManager.Task::(_decref) Task=`c51540af-6285-4150-933c-334fd392b809`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:12:19,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:19,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132588 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:21,317::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:21,318::__init__::232::IOProcess::(_processLogs) (6531) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:12:21,318::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:12:21,337::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412306 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:21,338::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:21,338::__init__::232::IOProcess::(_processLogs) (6532) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:21,340::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:21,340::__init__::232::IOProcess::(_processLogs) (6533) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:21,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:21,341::__init__::232::IOProcess::(_processLogs) (6534) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:21,342::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:21,342::__init__::232::IOProcess::(_processLogs) (6535) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:12:21,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:21,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132297 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:23,752::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:23,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:23,812::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:12:24,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:24,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132756 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:26,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:26,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134931 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:28,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:28,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133578 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:28,818::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:28,819::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:28,820::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:28,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`93026ed7-17bb-4fd2-b0b5-3f2d3b9aa292`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:28,867::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:12:28,867::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412306', 'lastCheck': '7.5', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:12:28,868::task::1191::Storage.TaskManager.Task::(prepare) Task=`93026ed7-17bb-4fd2-b0b5-3f2d3b9aa292`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412306', 'lastCheck': '7.5', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:12:28,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`93026ed7-17bb-4fd2-b0b5-3f2d3b9aa292`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:28,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:28,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:28,869::task::993::Storage.TaskManager.Task::(_decref) Task=`93026ed7-17bb-4fd2-b0b5-3f2d3b9aa292`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:12:28,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:29,093::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:12:29,094::task::595::Storage.TaskManager.Task::(_updateState) Task=`2173b1f9-a745-4315-ab8a-c465d87ebcef`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:29,094::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:12:29,108::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:29,109::task::1191::Storage.TaskManager.Task::(prepare) Task=`2173b1f9-a745-4315-ab8a-c465d87ebcef`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:29,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`2173b1f9-a745-4315-ab8a-c465d87ebcef`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:29,109::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:29,109::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:29,110::task::993::Storage.TaskManager.Task::(_decref) Task=`2173b1f9-a745-4315-ab8a-c465d87ebcef`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:12:29,121::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:12:29,122::task::595::Storage.TaskManager.Task::(_updateState) Task=`50732b9f-9389-4ec7-b62a-dd8da1b79538`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:12:29,122::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:12:29,123::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9551b33-13de-4b59-9077-dbbec7b252cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:12:29,123::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:12:29,123::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:12:29,124::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9551b33-13de-4b59-9077-dbbec7b252cf`::Granted request >Thread-34::DEBUG::2014-08-11 15:12:29,124::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`50732b9f-9389-4ec7-b62a-dd8da1b79538`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:12:29,124::task::993::Storage.TaskManager.Task::(_decref) Task=`50732b9f-9389-4ec7-b62a-dd8da1b79538`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:12:29,144::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:29,145::task::1191::Storage.TaskManager.Task::(prepare) Task=`50732b9f-9389-4ec7-b62a-dd8da1b79538`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:29,145::task::595::Storage.TaskManager.Task::(_updateState) Task=`50732b9f-9389-4ec7-b62a-dd8da1b79538`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:12:29,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:12:29,145::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:12:29,146::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:12:29,146::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:12:29,146::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:12:29,147::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:12:29,147::task::993::Storage.TaskManager.Task::(_decref) Task=`50732b9f-9389-4ec7-b62a-dd8da1b79538`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:12:30,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:30,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132197 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:31,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:31,345::__init__::232::IOProcess::(_processLogs) (6536) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:31,345::__init__::232::IOProcess::(_processLogs) (6536) Queuing response >Thread-16::DEBUG::2014-08-11 15:12:31,346::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:12:31,367::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00121502 s, 374 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:31,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:31,369::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:31,369::__init__::232::IOProcess::(_processLogs) (6537) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:31,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:31,371::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:31,371::__init__::232::IOProcess::(_processLogs) (6538) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:31,371::__init__::232::IOProcess::(_processLogs) (6538) Queuing response >Thread-14::DEBUG::2014-08-11 15:12:31,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:31,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:31,373::__init__::232::IOProcess::(_processLogs) (6539) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:31,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:31,377::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:31,377::__init__::232::IOProcess::(_processLogs) (6540) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:31,377::__init__::232::IOProcess::(_processLogs) (6540) Queuing response >Dummy-33::DEBUG::2014-08-11 15:12:32,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:32,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0284732 s, 36.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:33,827::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:33,829::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:33,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:12:34,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:34,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132837 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:36,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:36,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013191 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:38,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:38,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131901 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:38,920::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:38,921::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:38,922::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:39,163::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-113::DEBUG::2014-08-11 15:12:39,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`34fe976c-b76a-4c58-a376-178785b7547e`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:39,164::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:12:39,177::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:39,177::task::1191::Storage.TaskManager.Task::(prepare) Task=`34fe976c-b76a-4c58-a376-178785b7547e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:39,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`34fe976c-b76a-4c58-a376-178785b7547e`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:39,178::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:39,178::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:39,178::task::993::Storage.TaskManager.Task::(_decref) Task=`34fe976c-b76a-4c58-a376-178785b7547e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:12:39,188::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 15:12:39,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`def0be5a-5c30-4c43-8bca-bbbd0073dcaf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:12:39,189::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:12:39,190::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8aa76427-5c18-4de3-b9a9-b1270d92bf19`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:12:39,190::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:12:39,191::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:12:39,191::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8aa76427-5c18-4de3-b9a9-b1270d92bf19`::Granted request >Thread-34::DEBUG::2014-08-11 15:12:39,191::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`def0be5a-5c30-4c43-8bca-bbbd0073dcaf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:12:39,192::task::993::Storage.TaskManager.Task::(_decref) Task=`def0be5a-5c30-4c43-8bca-bbbd0073dcaf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:12:39,211::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:39,212::task::1191::Storage.TaskManager.Task::(prepare) Task=`def0be5a-5c30-4c43-8bca-bbbd0073dcaf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858136047616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:39,212::task::595::Storage.TaskManager.Task::(_updateState) Task=`def0be5a-5c30-4c43-8bca-bbbd0073dcaf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:12:39,212::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:12:39,212::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:12:39,213::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:12:39,213::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:12:39,213::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:12:39,214::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:12:39,214::task::993::Storage.TaskManager.Task::(_decref) Task=`def0be5a-5c30-4c43-8bca-bbbd0073dcaf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:12:40,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:40,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131567 s, 77.8 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:12:41,378::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:12:41,381::__init__::232::IOProcess::(_processLogs) (6541) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:12:41,388::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:12:41,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:41,389::__init__::232::IOProcess::(_processLogs) (6542) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 15:12:41,391::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:12:41,391::__init__::232::IOProcess::(_processLogs) (6543) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 15:12:41,393::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:12:41,394::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:41,394::__init__::232::IOProcess::(_processLogs) (6544) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:41,395::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:41,395::__init__::232::IOProcess::(_processLogs) (6545) Queuing response >Thread-16::DEBUG::2014-08-11 15:12:41,396::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:12:41,396::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:12:41,396::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:12:41,397::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:41,397::__init__::232::IOProcess::(_processLogs) (6546) Queuing response >Thread-16::DEBUG::2014-08-11 15:12:41,398::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:12:41,416::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423023 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:41,417::__init__::232::IOProcess::(_processLogs) (6547) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:41,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:41,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:41,420::__init__::232::IOProcess::(_processLogs) (6548) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:41,420::__init__::232::IOProcess::(_processLogs) (6548) Queuing response >Thread-14::DEBUG::2014-08-11 15:12:41,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:41,421::__init__::232::IOProcess::(_processLogs) (6549) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:41,421::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6549) Queuing response >Thread-14::DEBUG::2014-08-11 15:12:41,422::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:12:41,423::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:41,423::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:41,423::__init__::232::IOProcess::(_processLogs) (6550) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:41,423::__init__::232::IOProcess::(_processLogs) (6550) Queuing response >Dummy-33::DEBUG::2014-08-11 15:12:42,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:42,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132702 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:43,929::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:43,931::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:43,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`47fa2994-9fc6-4560-883c-f66b6b31843e`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:44,014::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:12:44,015::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423023', 'lastCheck': '2.6', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:12:44,017::task::1191::Storage.TaskManager.Task::(prepare) Task=`47fa2994-9fc6-4560-883c-f66b6b31843e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423023', 'lastCheck': '2.6', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:44,017::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:44,018::task::595::Storage.TaskManager.Task::(_updateState) Task=`47fa2994-9fc6-4560-883c-f66b6b31843e`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:44,019::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:44,021::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:44,021::task::993::Storage.TaskManager.Task::(_decref) Task=`47fa2994-9fc6-4560-883c-f66b6b31843e`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:12:44,044::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:12:44,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:44,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131877 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:46,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:46,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.021352 s, 48.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:48,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:48,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0326841 s, 31.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:49,027::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:49,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:49,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:49,228::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:12:49,229::task::595::Storage.TaskManager.Task::(_updateState) Task=`af293960-791f-4c1e-ae4b-344a22db6f82`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:49,229::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:12:49,243::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:49,243::task::1191::Storage.TaskManager.Task::(prepare) Task=`af293960-791f-4c1e-ae4b-344a22db6f82`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:49,243::task::595::Storage.TaskManager.Task::(_updateState) Task=`af293960-791f-4c1e-ae4b-344a22db6f82`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:49,244::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:49,244::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:49,244::task::993::Storage.TaskManager.Task::(_decref) Task=`af293960-791f-4c1e-ae4b-344a22db6f82`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:12:49,255::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:12:49,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`80ed210e-640f-482c-96a0-22bfada7ee33`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:12:49,256::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:12:49,257::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed465f9d-e96b-45c3-89d8-6aa0bdf0fec6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:12:49,257::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:12:49,257::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:12:49,257::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ed465f9d-e96b-45c3-89d8-6aa0bdf0fec6`::Granted request >Thread-34::DEBUG::2014-08-11 15:12:49,258::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`80ed210e-640f-482c-96a0-22bfada7ee33`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:12:49,258::task::993::Storage.TaskManager.Task::(_decref) Task=`80ed210e-640f-482c-96a0-22bfada7ee33`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:12:49,278::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858134671360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:49,278::task::1191::Storage.TaskManager.Task::(prepare) Task=`80ed210e-640f-482c-96a0-22bfada7ee33`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858134671360', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:49,279::task::595::Storage.TaskManager.Task::(_updateState) Task=`80ed210e-640f-482c-96a0-22bfada7ee33`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:12:49,279::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:12:49,279::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:12:49,279::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:12:49,280::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:12:49,280::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:12:49,280::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:12:49,281::task::993::Storage.TaskManager.Task::(_decref) Task=`80ed210e-640f-482c-96a0-22bfada7ee33`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:12:51,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:51,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132747 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:51,425::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:12:51,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:51,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:51,426::__init__::232::IOProcess::(_processLogs) (6551) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:51,426::__init__::232::IOProcess::(_processLogs) (6551) Queuing response >Thread-16::DEBUG::2014-08-11 15:12:51,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:12:51,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417823 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:12:51,447::__init__::232::IOProcess::(_processLogs) (6552) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:12:51,449::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:51,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:51,449::__init__::232::IOProcess::(_processLogs) (6553) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:51,450::__init__::232::IOProcess::(_processLogs) DEBUG|(6553) Queuing response >Thread-14::DEBUG::2014-08-11 15:12:51,451::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:12:51,451::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:51,452::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:51,452::__init__::232::IOProcess::(_processLogs) (6554) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:51,453::__init__::232::IOProcess::(_processLogs) (6554) Queuing response >Thread-14::DEBUG::2014-08-11 15:12:51,453::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:12:51,454::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:12:51,455::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:12:51,455::__init__::232::IOProcess::(_processLogs) (6555) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:12:51,455::__init__::232::IOProcess::(_processLogs) (6555) Queuing response >Dummy-33::DEBUG::2014-08-11 15:12:53,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:53,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157406 s, 65.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:54,035::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:54,036::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:54,113::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:12:55,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:55,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133494 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:12:57,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:57,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132858 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:59,120::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:59,121::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:12:59,122::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:12:59,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`24c70c00-cad4-4610-ad25-f82843238438`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:59,136::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:12:59,136::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417823', 'lastCheck': '7.7', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:12:59,137::task::1191::Storage.TaskManager.Task::(prepare) Task=`24c70c00-cad4-4610-ad25-f82843238438`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417823', 'lastCheck': '7.7', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:12:59,137::task::595::Storage.TaskManager.Task::(_updateState) Task=`24c70c00-cad4-4610-ad25-f82843238438`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:59,137::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:59,137::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:59,138::task::993::Storage.TaskManager.Task::(_decref) Task=`24c70c00-cad4-4610-ad25-f82843238438`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:12:59,157::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:12:59,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:12:59,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019368 s, 52.9 MB/s\n'; <rc> = 0 >Thread-113::DEBUG::2014-08-11 15:12:59,294::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:12:59,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`52440ace-68b5-4bc0-88d7-f84566f5b59e`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:12:59,298::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:12:59,320::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:59,320::task::1191::Storage.TaskManager.Task::(prepare) Task=`52440ace-68b5-4bc0-88d7-f84566f5b59e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:12:59,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`52440ace-68b5-4bc0-88d7-f84566f5b59e`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:12:59,321::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:12:59,321::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:12:59,322::task::993::Storage.TaskManager.Task::(_decref) Task=`52440ace-68b5-4bc0-88d7-f84566f5b59e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:12:59,332::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:12:59,333::task::595::Storage.TaskManager.Task::(_updateState) Task=`6942b0ec-2dcd-4119-ac23-03791393d519`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:12:59,333::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:12:59,334::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3aa64c9c-418f-4fc8-922f-b6adeff77b45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:12:59,334::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:12:59,334::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:12:59,335::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3aa64c9c-418f-4fc8-922f-b6adeff77b45`::Granted request >Thread-34::DEBUG::2014-08-11 15:12:59,335::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6942b0ec-2dcd-4119-ac23-03791393d519`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:12:59,335::task::993::Storage.TaskManager.Task::(_decref) Task=`6942b0ec-2dcd-4119-ac23-03791393d519`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:12:59,355::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858124709888', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:59,356::task::1191::Storage.TaskManager.Task::(prepare) Task=`6942b0ec-2dcd-4119-ac23-03791393d519`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858124709888', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:12:59,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`6942b0ec-2dcd-4119-ac23-03791393d519`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:12:59,356::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:12:59,357::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:12:59,357::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:12:59,357::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:12:59,357::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:12:59,358::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:12:59,358::task::993::Storage.TaskManager.Task::(_decref) Task=`6942b0ec-2dcd-4119-ac23-03791393d519`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:13:01,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:01,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133866 s, 76.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:01,456::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:01,457::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:13:01,458::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:13:01,458::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:01,477::__init__::232::IOProcess::(_processLogs) (6556) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:01,478::__init__::232::IOProcess::(_processLogs) (6556) Queuing response >Thread-16::DEBUG::2014-08-11 15:13:01,478::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000817889 s, 556 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:01,479::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:01,480::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:01,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:01,480::__init__::232::IOProcess::(_processLogs) (6557) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:01,481::__init__::232::IOProcess::(_processLogs) DEBUG|(6557) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:01,483::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:01,483::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:01,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:01,484::__init__::232::IOProcess::(_processLogs) (6558) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:01,484::__init__::232::IOProcess::(_processLogs) (6558) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:01,485::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:01,486::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:01,486::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:01,486::__init__::232::IOProcess::(_processLogs) (6559) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:01,486::__init__::232::IOProcess::(_processLogs) (6559) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:01,487::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:01,487::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:01,488::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:01,489::__init__::232::IOProcess::(_processLogs) (6560) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:01,489::__init__::232::IOProcess::(_processLogs) (6560) Queuing response >Dummy-33::DEBUG::2014-08-11 15:13:03,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:03,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132453 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:04,129::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:04,131::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:04,214::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:13:05,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:05,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134656 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:07,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:07,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132324 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:09,221::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:09,222::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:09,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:09,371::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:13:09,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`13923f38-0a12-4d66-a7d3-d3577b275702`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:09,372::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:13:09,385::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:09,385::task::1191::Storage.TaskManager.Task::(prepare) Task=`13923f38-0a12-4d66-a7d3-d3577b275702`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:09,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`13923f38-0a12-4d66-a7d3-d3577b275702`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:09,386::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:09,386::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:09,386::task::993::Storage.TaskManager.Task::(_decref) Task=`13923f38-0a12-4d66-a7d3-d3577b275702`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:13:09,396::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:13:09,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`0db3eca5-ce15-494d-ba88-8de81f84063e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:13:09,397::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:13:09,398::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ff55508d-2c3c-4588-81b6-be3843dc40fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:13:09,398::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:13:09,399::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:13:09,399::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ff55508d-2c3c-4588-81b6-be3843dc40fa`::Granted request >Thread-34::DEBUG::2014-08-11 15:13:09,399::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0db3eca5-ce15-494d-ba88-8de81f84063e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:13:09,400::task::993::Storage.TaskManager.Task::(_decref) Task=`0db3eca5-ce15-494d-ba88-8de81f84063e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:13:09,419::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:09,420::task::1191::Storage.TaskManager.Task::(prepare) Task=`0db3eca5-ce15-494d-ba88-8de81f84063e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:09,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`0db3eca5-ce15-494d-ba88-8de81f84063e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:13:09,420::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:13:09,420::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:13:09,421::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:13:09,421::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:13:09,421::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:13:09,421::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:13:09,422::task::993::Storage.TaskManager.Task::(_decref) Task=`0db3eca5-ce15-494d-ba88-8de81f84063e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:13:09,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:09,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131469 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:11,490::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:11,490::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:11,490::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:11,490::__init__::232::IOProcess::(_processLogs) (6561) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:11,491::__init__::232::IOProcess::(_processLogs) DEBUG|(6561) Queuing response >Thread-16::DEBUG::2014-08-11 15:13:11,492::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:13:11,511::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428011 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:11,513::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:11,513::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:11,513::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:11,514::__init__::232::IOProcess::(_processLogs) (6562) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:11,514::__init__::232::IOProcess::(_processLogs) DEBUG|(6562) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:11,516::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:11,516::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:11,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:11,517::__init__::232::IOProcess::(_processLogs) (6563) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:11,517::__init__::232::IOProcess::(_processLogs) (6563) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:11,518::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:11,518::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:11,519::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:11,519::__init__::232::IOProcess::(_processLogs) (6564) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:11,520::__init__::232::IOProcess::(_processLogs) (6564) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:11,520::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:11,521::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:11,522::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:11,522::__init__::232::IOProcess::(_processLogs) (6565) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:11,522::__init__::232::IOProcess::(_processLogs) (6565) Queuing response >Dummy-33::DEBUG::2014-08-11 15:13:11,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:11,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131771 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:13,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:13,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131542 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:14,230::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:14,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:14,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`da6dd6e8-ca5c-46c3-a858-673189e90cea`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:14,314::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:13:14,316::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428011', 'lastCheck': '2.8', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:13:14,318::task::1191::Storage.TaskManager.Task::(prepare) Task=`da6dd6e8-ca5c-46c3-a858-673189e90cea`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428011', 'lastCheck': '2.8', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:14,318::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:14,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`da6dd6e8-ca5c-46c3-a858-673189e90cea`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:14,320::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:14,322::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:14,323::task::993::Storage.TaskManager.Task::(_decref) Task=`da6dd6e8-ca5c-46c3-a858-673189e90cea`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:13:14,344::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:13:15,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:15,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132152 s, 77.5 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:13:16,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b8dbbe5-00ca-4be4-ae0a-3cb99de419c2`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:13:16,424::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:13:16,425::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:16,425::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:16,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:16,426::__init__::232::IOProcess::(_processLogs) (6566) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:13:16,426::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6566) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:16,427::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:16,428::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 15:13:16,428::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:13:16,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:13:16,430::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b8dbbe5-00ca-4be4-ae0a-3cb99de419c2`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:13:16,431::__init__::232::IOProcess::(_processLogs) (6567) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:13:16,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b8dbbe5-00ca-4be4-ae0a-3cb99de419c2`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:13:16,432::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:13:16,432::__init__::232::IOProcess::(_processLogs) (6567) Queuing response >Thread-55::DEBUG::2014-08-11 15:13:16,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:13:16,434::task::993::Storage.TaskManager.Task::(_decref) Task=`1b8dbbe5-00ca-4be4-ae0a-3cb99de419c2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:13:17,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:17,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132047 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:19,329::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:19,330::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:19,331::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:19,436::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:13:19,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0947d7a-d84c-46d3-990d-cbe59c836da3`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:19,437::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:13:19,450::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:19,451::task::1191::Storage.TaskManager.Task::(prepare) Task=`b0947d7a-d84c-46d3-990d-cbe59c836da3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:19,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`b0947d7a-d84c-46d3-990d-cbe59c836da3`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:19,451::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:19,451::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:19,452::task::993::Storage.TaskManager.Task::(_decref) Task=`b0947d7a-d84c-46d3-990d-cbe59c836da3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:13:19,465::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:13:19,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad59fd84-781c-47e8-aea1-22020977a51e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:13:19,466::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:13:19,467::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b4891ada-614d-4b05-b0b7-d86bb4b144d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:13:19,467::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:13:19,467::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:13:19,467::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b4891ada-614d-4b05-b0b7-d86bb4b144d6`::Granted request >Thread-34::DEBUG::2014-08-11 15:13:19,468::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ad59fd84-781c-47e8-aea1-22020977a51e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:13:19,468::task::993::Storage.TaskManager.Task::(_decref) Task=`ad59fd84-781c-47e8-aea1-22020977a51e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:13:19,488::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:19,488::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad59fd84-781c-47e8-aea1-22020977a51e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:19,489::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad59fd84-781c-47e8-aea1-22020977a51e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:13:19,489::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:13:19,489::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:13:19,489::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:13:19,490::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:13:19,490::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:13:19,490::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:13:19,490::task::993::Storage.TaskManager.Task::(_decref) Task=`ad59fd84-781c-47e8-aea1-22020977a51e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:13:19,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:19,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133249 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:21,523::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:21,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:21,524::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:13:21,525::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:13:21,526::__init__::232::IOProcess::(_processLogs) (6568) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:21,544::__init__::232::IOProcess::(_processLogs) (6568) Queuing response >Thread-16::DEBUG::2014-08-11 15:13:21,546::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426504 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:21,547::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:21,547::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:21,547::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:21,548::__init__::232::IOProcess::(_processLogs) (6569) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:21,548::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6569) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:21,550::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:21,550::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:21,550::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:21,551::__init__::232::IOProcess::(_processLogs) (6570) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:21,551::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6570) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:21,552::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:21,553::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:21,553::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:21,553::__init__::232::IOProcess::(_processLogs) (6571) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:21,554::__init__::232::IOProcess::(_processLogs) (6571) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:21,554::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:21,554::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:21,555::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:21,555::__init__::232::IOProcess::(_processLogs) (6572) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:21,555::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6572) Queuing response >Dummy-33::DEBUG::2014-08-11 15:13:21,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:21,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134382 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:23,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:23,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131752 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:24,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:24,336::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:24,415::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:13:25,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:26,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133925 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:28,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:28,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133061 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:29,420::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:29,421::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:29,422::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:29,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad615ee6-4848-4b89-9df0-9515ad90252f`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:29,434::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:13:29,434::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426504', 'lastCheck': '7.9', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:13:29,434::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad615ee6-4848-4b89-9df0-9515ad90252f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426504', 'lastCheck': '7.9', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:13:29,435::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad615ee6-4848-4b89-9df0-9515ad90252f`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:29,435::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:29,435::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:29,435::task::993::Storage.TaskManager.Task::(_decref) Task=`ad615ee6-4848-4b89-9df0-9515ad90252f`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:13:29,454::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:29,506::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:13:29,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`a28bb2fd-7809-451a-bbf8-7f94ea030cee`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:29,507::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:13:29,519::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:29,520::task::1191::Storage.TaskManager.Task::(prepare) Task=`a28bb2fd-7809-451a-bbf8-7f94ea030cee`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:29,520::task::595::Storage.TaskManager.Task::(_updateState) Task=`a28bb2fd-7809-451a-bbf8-7f94ea030cee`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:29,520::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:29,521::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:29,521::task::993::Storage.TaskManager.Task::(_decref) Task=`a28bb2fd-7809-451a-bbf8-7f94ea030cee`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:13:29,531::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:13:29,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4972127-af74-4f56-9be9-a42aec28701c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:13:29,532::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:13:29,533::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9783eb56-4941-454c-a153-4b57bee717f4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:13:29,533::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:13:29,534::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:13:29,534::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9783eb56-4941-454c-a153-4b57bee717f4`::Granted request >Thread-34::DEBUG::2014-08-11 15:13:29,534::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c4972127-af74-4f56-9be9-a42aec28701c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:13:29,535::task::993::Storage.TaskManager.Task::(_decref) Task=`c4972127-af74-4f56-9be9-a42aec28701c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:13:29,554::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:29,555::task::1191::Storage.TaskManager.Task::(prepare) Task=`c4972127-af74-4f56-9be9-a42aec28701c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:29,555::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4972127-af74-4f56-9be9-a42aec28701c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:13:29,556::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:13:29,556::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:13:29,556::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:13:29,557::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:13:29,557::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:13:29,557::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:13:29,557::task::993::Storage.TaskManager.Task::(_decref) Task=`c4972127-af74-4f56-9be9-a42aec28701c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:13:30,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:30,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013141 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:31,557::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:31,558::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:31,558::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:31,558::__init__::232::IOProcess::(_processLogs) (6573) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:31,558::__init__::232::IOProcess::(_processLogs) (6573) Queuing response >Thread-16::DEBUG::2014-08-11 15:13:31,559::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:13:31,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412629 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:31,580::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:31,580::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:31,580::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:31,581::__init__::232::IOProcess::(_processLogs) (6574) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:31,581::__init__::232::IOProcess::(_processLogs) (6574) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:31,583::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:31,583::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:31,584::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:31,584::__init__::232::IOProcess::(_processLogs) (6575) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:31,585::__init__::232::IOProcess::(_processLogs) (6575) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:31,585::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:31,585::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:31,586::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:31,586::__init__::232::IOProcess::(_processLogs) (6576) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:31,587::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|(6576) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:31,588::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:31,588::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:31,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:31,590::__init__::232::IOProcess::(_processLogs) (6577) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:31,590::__init__::232::IOProcess::(_processLogs) (6577) Queuing response >Dummy-33::DEBUG::2014-08-11 15:13:32,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:32,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270738 s, 37.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:34,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:34,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131982 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:34,430::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:34,436::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:34,441::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:13:36,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:36,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132115 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:38,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:38,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132087 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:39,452::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:39,456::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:39,515::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:39,570::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:13:39,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e6407d7-da89-4c5a-9953-312e570c525c`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:39,571::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:13:39,584::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:39,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e6407d7-da89-4c5a-9953-312e570c525c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:39,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e6407d7-da89-4c5a-9953-312e570c525c`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:39,585::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:39,585::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:39,585::task::993::Storage.TaskManager.Task::(_decref) Task=`9e6407d7-da89-4c5a-9953-312e570c525c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:13:39,596::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:13:39,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`78ab6aeb-6bb9-4420-9b32-92bfe862894c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:13:39,597::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:13:39,598::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a059f4e3-5e63-4921-a834-ad8e8e28b061`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:13:39,599::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:13:39,599::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:13:39,599::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a059f4e3-5e63-4921-a834-ad8e8e28b061`::Granted request >Thread-34::DEBUG::2014-08-11 15:13:39,600::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`78ab6aeb-6bb9-4420-9b32-92bfe862894c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:13:39,600::task::993::Storage.TaskManager.Task::(_decref) Task=`78ab6aeb-6bb9-4420-9b32-92bfe862894c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:13:39,620::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:39,620::task::1191::Storage.TaskManager.Task::(prepare) Task=`78ab6aeb-6bb9-4420-9b32-92bfe862894c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858120122368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:39,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`78ab6aeb-6bb9-4420-9b32-92bfe862894c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:13:39,621::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:13:39,621::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:13:39,621::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:13:39,622::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:13:39,622::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:13:39,622::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:13:39,622::task::993::Storage.TaskManager.Task::(_decref) Task=`78ab6aeb-6bb9-4420-9b32-92bfe862894c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:13:40,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:40,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01347 s, 76.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:41,590::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:41,590::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:41,591::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:41,591::__init__::232::IOProcess::(_processLogs) (6578) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:41,592::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|(6578) Queuing response >Thread-16::DEBUG::2014-08-11 15:13:41,593::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:13:41,613::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041195 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:41,614::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:41,614::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:41,615::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:41,615::__init__::232::IOProcess::(_processLogs) (6579) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:41,615::__init__::232::IOProcess::(_processLogs) (6579) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:41,616::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:41,617::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:41,618::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:41,618::__init__::232::IOProcess::(_processLogs) (6580) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:41,618::__init__::232::IOProcess::(_processLogs) (6580) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:41,618::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:41,619::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:41,620::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:41,620::__init__::232::IOProcess::(_processLogs) (6581) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:41,620::__init__::232::IOProcess::(_processLogs) (6581) Queuing response >Thread-14::DEBUG::2014-08-11 15:13:41,621::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:41,621::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:41,622::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:41,622::__init__::232::IOProcess::(_processLogs) (6582) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:41,623::__init__::232::IOProcess::(_processLogs) (6582) Queuing response >Dummy-33::DEBUG::2014-08-11 15:13:42,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:42,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132914 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:44,522::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:44,524::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:44,556::task::595::Storage.TaskManager.Task::(_updateState) Task=`48fd43ba-945b-4927-89c9-c67446fb5437`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:44,616::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:13:44,618::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041195', 'lastCheck': '3.0', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:44,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:44,620::task::1191::Storage.TaskManager.Task::(prepare) Task=`48fd43ba-945b-4927-89c9-c67446fb5437`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041195', 'lastCheck': '3.0', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:13:44,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`48fd43ba-945b-4927-89c9-c67446fb5437`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:44,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:44,623::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:44,623::task::993::Storage.TaskManager.Task::(_decref) Task=`48fd43ba-945b-4927-89c9-c67446fb5437`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:13:44,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:13:44,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:44,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131986 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:46,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:46,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013209 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:48,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:48,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135188 s, 75.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:49,628::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:49,629::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:49,630::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:49,635::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:13:49,635::task::595::Storage.TaskManager.Task::(_updateState) Task=`e086a7b4-55d3-4968-ab78-bd83efe955bc`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:49,636::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::INFO::2014-08-11 15:13:49,648::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:49,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`e086a7b4-55d3-4968-ab78-bd83efe955bc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:49,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`e086a7b4-55d3-4968-ab78-bd83efe955bc`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:49,649::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:13:49,649::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:49,650::task::993::Storage.TaskManager.Task::(_decref) Task=`e086a7b4-55d3-4968-ab78-bd83efe955bc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:13:49,659::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:13:49,660::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4e66187-7c7b-4c41-934b-f71e6ecf0371`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:13:49,660::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:13:49,661::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`07398f51-e14b-47fc-8b41-ad9a23959724`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:13:49,661::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:13:49,662::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:13:49,662::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`07398f51-e14b-47fc-8b41-ad9a23959724`::Granted request >Thread-34::DEBUG::2014-08-11 15:13:49,662::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b4e66187-7c7b-4c41-934b-f71e6ecf0371`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:13:49,663::task::993::Storage.TaskManager.Task::(_decref) Task=`b4e66187-7c7b-4c41-934b-f71e6ecf0371`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:13:49,683::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858087157760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:49,683::task::1191::Storage.TaskManager.Task::(prepare) Task=`b4e66187-7c7b-4c41-934b-f71e6ecf0371`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1858087157760', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:49,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4e66187-7c7b-4c41-934b-f71e6ecf0371`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:13:49,683::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:13:49,684::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:13:49,684::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:13:49,684::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:13:49,685::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:13:49,685::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:13:49,685::task::993::Storage.TaskManager.Task::(_decref) Task=`b4e66187-7c7b-4c41-934b-f71e6ecf0371`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:13:50,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:13:51,623::__init__::232::IOProcess::(_processLogs) (6574) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:13:51,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:51,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:51,624::__init__::232::IOProcess::(_processLogs) (6583) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:13:52,386::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:52,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.51405 s, 676 kB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:13:52,412::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000655715 s, 694 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:13:52,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:52,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:52,421::__init__::232::IOProcess::(_processLogs) (6584) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:13:52,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:52,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:52,427::__init__::232::IOProcess::(_processLogs) (6585) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:52,430::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:52,431::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:52,432::__init__::232::IOProcess::(_processLogs) (6586) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:13:52,435::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:13:52,436::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:13:52,437::__init__::232::IOProcess::(_processLogs) (6587) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:13:54,456::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:54,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131806 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:54,637::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:54,639::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:54,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:13:56,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:56,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013403 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:13:58,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:13:58,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138766 s, 73.8 MB/s\n'; <rc> = 0 >Thread-113::DEBUG::2014-08-11 15:13:59,697::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:13:59,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`35222628-fd2b-44e4-b22c-37a11bd86ecc`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:13:59,716::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:59,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 15:13:59,728::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:49857 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:59,729::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 15:13:59,729::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:49857 >Thread-113::INFO::2014-08-11 15:13:59,730::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:13:59,732::task::1191::Storage.TaskManager.Task::(prepare) Task=`35222628-fd2b-44e4-b22c-37a11bd86ecc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Detector thread::DEBUG::2014-08-11 15:13:59,732::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:49857 >Thread-113::DEBUG::2014-08-11 15:13:59,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`35222628-fd2b-44e4-b22c-37a11bd86ecc`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:13:59,735::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Detector thread::DEBUG::2014-08-11 15:13:59,735::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 49857) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:13:59,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:13:59,737::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:13:59,739::task::993::Storage.TaskManager.Task::(_decref) Task=`35222628-fd2b-44e4-b22c-37a11bd86ecc`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:13:59,742::task::595::Storage.TaskManager.Task::(_updateState) Task=`f11987ca-eff0-4ef2-9d51-10c0eaea85d6`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:13:59,743::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:13:59,743::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000655715', 'lastCheck': '7.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:13:59,743::task::1191::Storage.TaskManager.Task::(prepare) Task=`f11987ca-eff0-4ef2-9d51-10c0eaea85d6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000655715', 'lastCheck': '7.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:13:59,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`f11987ca-eff0-4ef2-9d51-10c0eaea85d6`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:13:59,744::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:13:59,744::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:13:59,744::task::993::Storage.TaskManager.Task::(_decref) Task=`f11987ca-eff0-4ef2-9d51-10c0eaea85d6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:13:59,750::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:13:59,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`952d2fed-221b-48ce-a4e0-774893a2716b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:13:59,751::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:13:59,751::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c83b6d25-9acb-400b-a8a6-86a0c21bd034`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:13:59,751::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:13:59,752::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:13:59,752::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c83b6d25-9acb-400b-a8a6-86a0c21bd034`::Granted request >Thread-34::DEBUG::2014-08-11 15:13:59,753::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`952d2fed-221b-48ce-a4e0-774893a2716b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:13:59,753::task::993::Storage.TaskManager.Task::(_decref) Task=`952d2fed-221b-48ce-a4e0-774893a2716b`::ref 1 aborting False >Thread-115::DEBUG::2014-08-11 15:13:59,766::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:13:59,773::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857665630208', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:59,773::task::1191::Storage.TaskManager.Task::(prepare) Task=`952d2fed-221b-48ce-a4e0-774893a2716b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857665630208', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:13:59,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`952d2fed-221b-48ce-a4e0-774893a2716b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:13:59,774::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:13:59,774::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:13:59,774::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:13:59,775::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:13:59,775::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:13:59,775::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:13:59,775::task::993::Storage.TaskManager.Task::(_decref) Task=`952d2fed-221b-48ce-a4e0-774893a2716b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:14:00,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:00,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181589 s, 56.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:02,442::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:02,443::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:02,445::__init__::232::IOProcess::(_processLogs) (6588) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:14:02,447::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:14:02,469::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000620492 s, 733 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:02,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:02,473::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:02,475::__init__::232::IOProcess::(_processLogs) (6589) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:02,478::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:02,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:02,481::__init__::232::IOProcess::(_processLogs) (6590) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:02,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:02,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:02,487::__init__::232::IOProcess::(_processLogs) (6591) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:02,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:02,492::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:02,494::__init__::232::IOProcess::(_processLogs) (6592) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:14:02,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:02,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271717 s, 37.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:04,747::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:04,748::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:04,816::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:14:04,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:04,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133008 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:07,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:07,041::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131471 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:09,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:09,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132307 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:14:09,788::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:14:09,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0a4dfdb-2ea9-4790-aa2a-09f3eb1ac62a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:14:09,816::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:09,824::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:09,825::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:09,826::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:14:09,829::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:14:09,830::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0a4dfdb-2ea9-4790-aa2a-09f3eb1ac62a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:14:09,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0a4dfdb-2ea9-4790-aa2a-09f3eb1ac62a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:14:09,830::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:14:09,830::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:14:09,830::task::993::Storage.TaskManager.Task::(_decref) Task=`d0a4dfdb-2ea9-4790-aa2a-09f3eb1ac62a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:14:09,842::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:14:09,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`1771ed40-ed48-47a4-a850-27bace5e4a9a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:14:09,843::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:14:09,844::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c201f369-e58b-47b3-8ecf-7c719afe4a9c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:14:09,844::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:14:09,845::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:14:09,845::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c201f369-e58b-47b3-8ecf-7c719afe4a9c`::Granted request >Thread-34::DEBUG::2014-08-11 15:14:09,845::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1771ed40-ed48-47a4-a850-27bace5e4a9a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:14:09,846::task::993::Storage.TaskManager.Task::(_decref) Task=`1771ed40-ed48-47a4-a850-27bace5e4a9a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:14:09,866::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857661435904', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:09,866::task::1191::Storage.TaskManager.Task::(prepare) Task=`1771ed40-ed48-47a4-a850-27bace5e4a9a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857661435904', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:09,866::task::595::Storage.TaskManager.Task::(_updateState) Task=`1771ed40-ed48-47a4-a850-27bace5e4a9a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:14:09,867::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:14:09,867::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:14:09,867::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:14:09,867::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:14:09,868::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:14:09,868::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:14:09,868::task::993::Storage.TaskManager.Task::(_decref) Task=`1771ed40-ed48-47a4-a850-27bace5e4a9a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:14:11,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:11,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131899 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:12,498::__init__::232::IOProcess::(_processLogs) (6593) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:14:12,499::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:14:12,518::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422371 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:12,519::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:12,519::__init__::232::IOProcess::(_processLogs) (6594) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:12,521::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:12,521::__init__::232::IOProcess::(_processLogs) (6595) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:12,522::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:12,522::__init__::232::IOProcess::(_processLogs) (6596) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:12,523::__init__::232::IOProcess::(_processLogs) (6596) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:12,523::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:12,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:12,524::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:12,525::__init__::232::IOProcess::(_processLogs) (6597) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:12,526::__init__::232::IOProcess::(_processLogs) (6597) Queuing response >Dummy-33::DEBUG::2014-08-11 15:14:13,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:13,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132457 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:14,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:14,834::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:14,835::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:14:14,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0552256-8c3d-4144-a5a2-64d5873f3780`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:14:14,865::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:14:14,865::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422371', 'lastCheck': '2.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:14:14,866::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0552256-8c3d-4144-a5a2-64d5873f3780`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422371', 'lastCheck': '2.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:14:14,866::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0552256-8c3d-4144-a5a2-64d5873f3780`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:14:14,866::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:14:14,866::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:14:14,866::task::993::Storage.TaskManager.Task::(_decref) Task=`a0552256-8c3d-4144-a5a2-64d5873f3780`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:14:14,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:14:15,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:15,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132882 s, 77.1 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:14:16,558::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7643831-cf1b-4d3d-b258-46cc4bb1c2de`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:14:16,558::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:14:16,559::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:16,560::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:16,560::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:16,560::__init__::232::IOProcess::(_processLogs) (6598) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:14:16,560::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6598) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:16,561::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:16,562::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 15:14:16,562::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:14:16,563::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:14:16,564::task::1191::Storage.TaskManager.Task::(prepare) Task=`a7643831-cf1b-4d3d-b258-46cc4bb1c2de`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:14:16,565::__init__::232::IOProcess::(_processLogs) (6599) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:14:16,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7643831-cf1b-4d3d-b258-46cc4bb1c2de`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:14:16,566::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:14:16,566::__init__::232::IOProcess::(_processLogs) (6599) Queuing response >Thread-55::DEBUG::2014-08-11 15:14:16,567::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:14:16,568::task::993::Storage.TaskManager.Task::(_decref) Task=`a7643831-cf1b-4d3d-b258-46cc4bb1c2de`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:14:17,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:17,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132522 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:19,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:19,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134017 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:19,844::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:19,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:14:19,882::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:14:19,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac1c7021-325b-41b9-b086-59c85bfe7fb5`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:14:19,918::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:19,919::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:14:19,934::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:14:19,934::task::1191::Storage.TaskManager.Task::(prepare) Task=`ac1c7021-325b-41b9-b086-59c85bfe7fb5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:14:19,934::task::595::Storage.TaskManager.Task::(_updateState) Task=`ac1c7021-325b-41b9-b086-59c85bfe7fb5`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:14:19,934::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:14:19,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:14:19,935::task::993::Storage.TaskManager.Task::(_decref) Task=`ac1c7021-325b-41b9-b086-59c85bfe7fb5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:14:19,948::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:14:19,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`d13e3ae9-816e-4185-83a9-19caf7bbbd80`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:14:19,949::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:14:19,950::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2e4ee62d-0641-41aa-8e50-84574d52735d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:14:19,950::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:14:19,950::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:14:19,951::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2e4ee62d-0641-41aa-8e50-84574d52735d`::Granted request >Thread-34::DEBUG::2014-08-11 15:14:19,951::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d13e3ae9-816e-4185-83a9-19caf7bbbd80`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:14:19,951::task::993::Storage.TaskManager.Task::(_decref) Task=`d13e3ae9-816e-4185-83a9-19caf7bbbd80`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:14:19,971::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857652391936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:19,972::task::1191::Storage.TaskManager.Task::(prepare) Task=`d13e3ae9-816e-4185-83a9-19caf7bbbd80`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857652391936', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:19,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`d13e3ae9-816e-4185-83a9-19caf7bbbd80`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:14:19,972::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:14:19,972::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:14:19,973::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:14:19,973::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:14:19,973::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:14:19,973::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:14:19,974::task::993::Storage.TaskManager.Task::(_decref) Task=`d13e3ae9-816e-4185-83a9-19caf7bbbd80`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:14:21,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:21,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013291 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:22,527::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:22,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:22,528::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:14:22,529::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:14:22,529::__init__::232::IOProcess::(_processLogs) (6600) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:22,548::__init__::232::IOProcess::(_processLogs) (6600) Queuing response >Thread-16::DEBUG::2014-08-11 15:14:22,549::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000492405 s, 924 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:22,550::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:22,551::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:22,551::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:22,551::__init__::232::IOProcess::(_processLogs) (6601) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:22,552::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6601) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:22,553::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:22,554::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:22,554::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:22,554::__init__::232::IOProcess::(_processLogs) (6602) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:22,555::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6602) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:22,556::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:22,556::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:22,557::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:22,557::__init__::232::IOProcess::(_processLogs) (6603) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:22,557::__init__::232::IOProcess::(_processLogs) (6603) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:22,558::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:22,558::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:22,558::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:22,559::__init__::232::IOProcess::(_processLogs) (6604) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:22,559::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6604) Queuing response >Dummy-33::DEBUG::2014-08-11 15:14:23,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:23,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131233 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:24,926::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:24,927::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:25,016::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:14:25,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:25,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132482 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:27,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:27,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133308 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:29,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:29,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132593 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:14:29,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`e40df342-fa2a-4918-9730-e75d8dd506a2`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:14:29,977::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::DEBUG::2014-08-11 15:14:29,988::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::INFO::2014-08-11 15:14:30,016::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000492405', 'lastCheck': '7.5', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:14:30,018::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee2af6a4-68a0-4b4d-b263-a9085d2d5a1d`::moving from state init -> state preparing >Thread-115::DEBUG::2014-08-11 15:14:30,018::task::1191::Storage.TaskManager.Task::(prepare) Task=`e40df342-fa2a-4918-9730-e75d8dd506a2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000492405', 'lastCheck': '7.5', 'valid': True}} >Thread-113::INFO::2014-08-11 15:14:30,019::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::DEBUG::2014-08-11 15:14:30,020::task::595::Storage.TaskManager.Task::(_updateState) Task=`e40df342-fa2a-4918-9730-e75d8dd506a2`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:14:30,021::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:14:30,021::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:14:30,021::task::993::Storage.TaskManager.Task::(_decref) Task=`e40df342-fa2a-4918-9730-e75d8dd506a2`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:30,027::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::INFO::2014-08-11 15:14:30,036::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:14:30,037::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee2af6a4-68a0-4b4d-b263-a9085d2d5a1d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:14:30,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee2af6a4-68a0-4b4d-b263-a9085d2d5a1d`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:14:30,037::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:14:30,038::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:14:30,038::task::993::Storage.TaskManager.Task::(_decref) Task=`ee2af6a4-68a0-4b4d-b263-a9085d2d5a1d`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:30,039::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:30,040::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:14:30,051::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:14:30,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c487546-0b13-41c9-8acf-91f0e87db690`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:14:30,052::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:14:30,053::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`600f963b-1370-4934-84c8-6411992466c1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:14:30,053::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:14:30,053::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:14:30,053::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`600f963b-1370-4934-84c8-6411992466c1`::Granted request >Thread-34::DEBUG::2014-08-11 15:14:30,054::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1c487546-0b13-41c9-8acf-91f0e87db690`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:14:30,054::task::993::Storage.TaskManager.Task::(_decref) Task=`1c487546-0b13-41c9-8acf-91f0e87db690`::ref 1 aborting False >Thread-113::DEBUG::2014-08-11 15:14:30,063::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:14:30,092::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857558347776', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:30,093::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c487546-0b13-41c9-8acf-91f0e87db690`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857558347776', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:30,093::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c487546-0b13-41c9-8acf-91f0e87db690`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:14:30,093::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:14:30,093::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:14:30,094::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:14:30,094::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:14:30,094::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:14:30,095::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:14:30,095::task::993::Storage.TaskManager.Task::(_decref) Task=`1c487546-0b13-41c9-8acf-91f0e87db690`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:14:31,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:31,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0542291 s, 18.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:32,562::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:32,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:32,565::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:32,567::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6605) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:14:32,568::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:14:32,569::__init__::232::IOProcess::(_processLogs) (6605) Queuing response >Thread-16::DEBUG::2014-08-11 15:14:32,597::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00752363 s, 60.5 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:32,600::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:32,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:32,603::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:32,604::__init__::232::IOProcess::(_processLogs) (6606) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:32,606::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6606) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:32,609::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:32,610::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:32,611::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:32,613::__init__::232::IOProcess::(_processLogs) (6607) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:32,614::__init__::232::IOProcess::(_processLogs) (6607) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:32,616::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:32,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:32,620::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:32,621::__init__::232::IOProcess::(_processLogs) (6608) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:32,624::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6608) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:32,626::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:32,628::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:32,630::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:32,631::__init__::232::IOProcess::(_processLogs) (6609) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:32,632::__init__::232::IOProcess::(_processLogs) (6609) Queuing response >Dummy-33::DEBUG::2014-08-11 15:14:33,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:33,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132695 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:35,048::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:35,049::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:35,116::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:14:35,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:35,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132665 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:37,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:37,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131629 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:39,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:39,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135088 s, 75.8 MB/s\n'; <rc> = 0 >Thread-113::DEBUG::2014-08-11 15:14:40,107::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:14:40,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c6e4943-3aa9-4a08-8fc9-0b5b577730dd`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:14:40,116::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:40,123::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:40,125::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:40,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::INFO::2014-08-11 15:14:40,129::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:14:40,129::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c6e4943-3aa9-4a08-8fc9-0b5b577730dd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:14:40,129::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c6e4943-3aa9-4a08-8fc9-0b5b577730dd`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:14:40,130::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:14:40,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:14:40,130::task::993::Storage.TaskManager.Task::(_decref) Task=`9c6e4943-3aa9-4a08-8fc9-0b5b577730dd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:14:40,140::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:14:40,141::task::595::Storage.TaskManager.Task::(_updateState) Task=`98b8ecbc-df3d-44b1-b763-6abb06099a29`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:14:40,141::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:14:40,142::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3d4647c7-64b9-472b-b4dc-1171adb0a69c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:14:40,142::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:14:40,142::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:14:40,143::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3d4647c7-64b9-472b-b4dc-1171adb0a69c`::Granted request >Thread-34::DEBUG::2014-08-11 15:14:40,143::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`98b8ecbc-df3d-44b1-b763-6abb06099a29`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:14:40,143::task::993::Storage.TaskManager.Task::(_decref) Task=`98b8ecbc-df3d-44b1-b763-6abb06099a29`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:14:40,164::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857497661440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:40,164::task::1191::Storage.TaskManager.Task::(prepare) Task=`98b8ecbc-df3d-44b1-b763-6abb06099a29`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857497661440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:40,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`98b8ecbc-df3d-44b1-b763-6abb06099a29`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:14:40,164::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:14:40,165::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:14:40,165::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:14:40,165::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:14:40,165::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:14:40,166::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:14:40,166::task::993::Storage.TaskManager.Task::(_decref) Task=`98b8ecbc-df3d-44b1-b763-6abb06099a29`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:14:41,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:42,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132431 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:42,629::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:42,629::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:42,629::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:42,630::__init__::232::IOProcess::(_processLogs) (6610) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:42,630::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6610) Queuing response >Thread-16::DEBUG::2014-08-11 15:14:42,631::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:14:42,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422229 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:42,652::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:42,652::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:42,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:42,653::__init__::232::IOProcess::(_processLogs) (6611) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:42,654::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6611) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:42,655::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:42,656::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:42,656::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:42,657::__init__::232::IOProcess::(_processLogs) (6612) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:42,657::__init__::232::IOProcess::(_processLogs) (6612) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:42,657::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:42,658::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:42,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:42,659::__init__::232::IOProcess::(_processLogs) (6613) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:42,659::__init__::232::IOProcess::(_processLogs) (6613) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:42,659::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:42,660::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:42,660::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:42,661::__init__::232::IOProcess::(_processLogs) (6614) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:42,662::__init__::232::IOProcess::(_processLogs) (6614) Queuing response >Dummy-33::DEBUG::2014-08-11 15:14:44,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:44,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013291 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:45,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:45,134::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:14:45,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf4d152-b850-4f86-afc7-06e7c4c5f8c6`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:14:45,219::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:14:45,220::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422229', 'lastCheck': '2.6', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:14:45,221::task::1191::Storage.TaskManager.Task::(prepare) Task=`daf4d152-b850-4f86-afc7-06e7c4c5f8c6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422229', 'lastCheck': '2.6', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:14:45,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`daf4d152-b850-4f86-afc7-06e7c4c5f8c6`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:45,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:14:45,223::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:14:45,224::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:14:45,225::task::993::Storage.TaskManager.Task::(_decref) Task=`daf4d152-b850-4f86-afc7-06e7c4c5f8c6`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:14:45,248::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:14:46,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:46,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132995 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:48,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:48,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133244 s, 76.9 MB/s\n'; <rc> = 0 >Thread-113::DEBUG::2014-08-11 15:14:50,181::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::DEBUG::2014-08-11 15:14:50,218::task::595::Storage.TaskManager.Task::(_updateState) Task=`10774e07-6ce8-4e3b-951c-0b76af3e92fd`::moving from state init -> state preparing >Dummy-33::DEBUG::2014-08-11 15:14:50,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-113::INFO::2014-08-11 15:14:50,219::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:50,239::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:50,241::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:50,242::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::INFO::2014-08-11 15:14:50,250::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:14:50,251::task::1191::Storage.TaskManager.Task::(prepare) Task=`10774e07-6ce8-4e3b-951c-0b76af3e92fd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-113::DEBUG::2014-08-11 15:14:50,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`10774e07-6ce8-4e3b-951c-0b76af3e92fd`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:14:50,251::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:14:50,251::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:14:50,252::task::993::Storage.TaskManager.Task::(_decref) Task=`10774e07-6ce8-4e3b-951c-0b76af3e92fd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:14:50,262::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:14:50,263::task::595::Storage.TaskManager.Task::(_updateState) Task=`986b3c70-1e2d-4759-b372-aa501fd169f3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:14:50,264::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:14:50,267::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`00726178-1608-4212-8e92-7e3ac9cf87e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:14:50,269::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:14:50,270::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:14:50,272::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`00726178-1608-4212-8e92-7e3ac9cf87e3`::Granted request >Dummy-33::DEBUG::2014-08-11 15:14:50,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208064 s, 49.2 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 15:14:50,275::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`986b3c70-1e2d-4759-b372-aa501fd169f3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:14:50,276::task::993::Storage.TaskManager.Task::(_decref) Task=`986b3c70-1e2d-4759-b372-aa501fd169f3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:14:50,327::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857322156032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:50,328::task::1191::Storage.TaskManager.Task::(prepare) Task=`986b3c70-1e2d-4759-b372-aa501fd169f3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857322156032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:14:50,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`986b3c70-1e2d-4759-b372-aa501fd169f3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:14:50,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:14:50,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:14:50,329::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:14:50,329::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:14:50,329::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:14:50,329::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:14:50,330::task::993::Storage.TaskManager.Task::(_decref) Task=`986b3c70-1e2d-4759-b372-aa501fd169f3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:14:52,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:52,360::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131768 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:52,663::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:52,664::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:14:52,665::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:14:52,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:52,684::__init__::232::IOProcess::(_processLogs) (6615) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:52,685::__init__::232::IOProcess::(_processLogs) (6615) Queuing response >Thread-16::DEBUG::2014-08-11 15:14:52,686::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000508957 s, 894 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:14:52,687::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:52,687::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:52,688::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:52,688::__init__::232::IOProcess::(_processLogs) (6616) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:14:52,689::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6616) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:52,690::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:52,690::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:52,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:52,691::__init__::232::IOProcess::(_processLogs) (6617) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:52,692::__init__::232::IOProcess::(_processLogs) (6617) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:52,692::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:52,692::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:52,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:52,693::__init__::232::IOProcess::(_processLogs) (6618) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:52,694::__init__::232::IOProcess::(_processLogs) (6618) Queuing response >Thread-14::DEBUG::2014-08-11 15:14:52,694::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:14:52,695::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:14:52,695::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:14:52,696::__init__::232::IOProcess::(_processLogs) (6619) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:14:52,697::__init__::232::IOProcess::(_processLogs) (6619) Queuing response >Dummy-33::DEBUG::2014-08-11 15:14:54,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:54,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131977 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:55,249::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:55,250::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:14:55,318::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:14:56,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:56,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133675 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:14:58,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:14:58,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135603 s, 75.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:00,325::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:15:00,336::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c161034-33fc-437a-a227-9e37ba5c3929`::moving from state init -> state preparing >Thread-115::DEBUG::2014-08-11 15:15:00,344::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-113::INFO::2014-08-11 15:15:00,420::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::DEBUG::2014-08-11 15:15:00,422::task::595::Storage.TaskManager.Task::(_updateState) Task=`dfb7494d-56ba-4551-8e24-d0543341c853`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:15:00,422::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000508957', 'lastCheck': '7.7', 'valid': True}} >Thread-115::INFO::2014-08-11 15:15:00,424::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-113::DEBUG::2014-08-11 15:15:00,425::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c161034-33fc-437a-a227-9e37ba5c3929`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000508957', 'lastCheck': '7.7', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:15:00,426::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c161034-33fc-437a-a227-9e37ba5c3929`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:15:00,428::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:15:00,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:00,427::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:15:00,428::task::993::Storage.TaskManager.Task::(_decref) Task=`0c161034-33fc-437a-a227-9e37ba5c3929`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:00,432::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:15:00,441::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:00,441::task::1191::Storage.TaskManager.Task::(prepare) Task=`dfb7494d-56ba-4551-8e24-d0543341c853`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:00,441::task::595::Storage.TaskManager.Task::(_updateState) Task=`dfb7494d-56ba-4551-8e24-d0543341c853`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:00,441::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:00,442::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:15:00,442::task::993::Storage.TaskManager.Task::(_decref) Task=`dfb7494d-56ba-4551-8e24-d0543341c853`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:15:00,454::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:15:00,454::task::595::Storage.TaskManager.Task::(_updateState) Task=`49f6e25d-bd53-4f99-bf64-353b7d0303e3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:15:00,455::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:15:00,455::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cc52260e-2329-4f21-8254-626b51b9817d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:15:00,456::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:15:00,456::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:15:00,456::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cc52260e-2329-4f21-8254-626b51b9817d`::Granted request >Thread-34::DEBUG::2014-08-11 15:15:00,457::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`49f6e25d-bd53-4f99-bf64-353b7d0303e3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:15:00,457::task::993::Storage.TaskManager.Task::(_decref) Task=`49f6e25d-bd53-4f99-bf64-353b7d0303e3`::ref 1 aborting False >Thread-115::DEBUG::2014-08-11 15:15:00,469::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:15:00,484::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857179942912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:00,485::task::1191::Storage.TaskManager.Task::(prepare) Task=`49f6e25d-bd53-4f99-bf64-353b7d0303e3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857179942912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:00,485::task::595::Storage.TaskManager.Task::(_updateState) Task=`49f6e25d-bd53-4f99-bf64-353b7d0303e3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:15:00,485::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:15:00,485::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:15:00,486::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:15:00,486::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:15:00,486::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:15:00,486::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:15:00,487::task::993::Storage.TaskManager.Task::(_decref) Task=`49f6e25d-bd53-4f99-bf64-353b7d0303e3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:15:00,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:00,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133483 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:02,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:02,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279358 s, 36.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:02,698::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:02,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:02,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:02,702::__init__::232::IOProcess::(_processLogs) (6620) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:02,704::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6620) Queuing response >Thread-16::DEBUG::2014-08-11 15:15:02,707::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:15:02,730::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000544656 s, 835 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:02,735::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:02,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:02,739::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:02,742::__init__::232::IOProcess::(_processLogs) (6621) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:02,746::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6621) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:02,748::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:02,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:02,751::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:02,753::__init__::232::IOProcess::(_processLogs) (6622) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:02,754::__init__::232::IOProcess::(_processLogs) (6622) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:02,759::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:02,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:02,765::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:02,767::__init__::232::IOProcess::(_processLogs) (6623) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:02,770::__init__::232::IOProcess::(_processLogs) (6623) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:02,774::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:02,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:02,777::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:02,779::__init__::232::IOProcess::(_processLogs) (6624) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:02,780::__init__::232::IOProcess::(_processLogs) (6624) Queuing response >Dummy-33::DEBUG::2014-08-11 15:15:04,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:04,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131972 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:05,442::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:05,443::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:05,519::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:15:06,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:07,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132114 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:09,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:09,072::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132643 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:15:10,499::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:15:10,500::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4f56cb8-841b-4390-a537-07b3cb95e573`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:15:10,518::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:10,526::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:10,528::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:10,529::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:15:10,531::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:10,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`c4f56cb8-841b-4390-a537-07b3cb95e573`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:10,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4f56cb8-841b-4390-a537-07b3cb95e573`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:10,532::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:10,532::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:15:10,533::task::993::Storage.TaskManager.Task::(_decref) Task=`c4f56cb8-841b-4390-a537-07b3cb95e573`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:15:10,543::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:15:10,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff195b5c-3a7c-4d1a-a469-36d216724c69`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:15:10,543::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:15:10,544::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`80e62991-bff5-4927-8c49-9036aff0af69`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:15:10,544::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:15:10,545::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:15:10,545::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`80e62991-bff5-4927-8c49-9036aff0af69`::Granted request >Thread-34::DEBUG::2014-08-11 15:15:10,546::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ff195b5c-3a7c-4d1a-a469-36d216724c69`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:15:10,546::task::993::Storage.TaskManager.Task::(_decref) Task=`ff195b5c-3a7c-4d1a-a469-36d216724c69`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:15:10,566::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857052344320', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:10,566::task::1191::Storage.TaskManager.Task::(prepare) Task=`ff195b5c-3a7c-4d1a-a469-36d216724c69`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1857052344320', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:10,566::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff195b5c-3a7c-4d1a-a469-36d216724c69`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:15:10,566::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:15:10,567::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:15:10,567::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:15:10,567::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:15:10,567::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:15:10,568::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:15:10,568::task::993::Storage.TaskManager.Task::(_decref) Task=`ff195b5c-3a7c-4d1a-a469-36d216724c69`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:15:11,096::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:11,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133504 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:12,784::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:12,785::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:12,785::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:12,785::__init__::232::IOProcess::(_processLogs) (6625) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:12,786::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6625) Queuing response >Thread-16::DEBUG::2014-08-11 15:15:12,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:15:12,806::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420263 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:12,807::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:12,808::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:12,808::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:12,808::__init__::232::IOProcess::(_processLogs) (6626) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:12,809::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6626) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:12,811::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:12,811::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:12,811::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:12,812::__init__::232::IOProcess::(_processLogs) (6627) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:12,812::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6627) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:12,813::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:12,813::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:12,813::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:12,814::__init__::232::IOProcess::(_processLogs) (6628) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:12,814::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6628) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:12,815::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:12,815::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:12,816::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:12,816::__init__::232::IOProcess::(_processLogs) (6629) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:12,816::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6629) Queuing response >Dummy-33::DEBUG::2014-08-11 15:15:13,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:13,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137919 s, 74.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:15,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:15,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168601 s, 60.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:15,536::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:15,537::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:15:15,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0ed7114-6873-49bc-90ec-3d515963cfe3`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:15:15,621::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:15:15,623::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420263', 'lastCheck': '2.8', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:15:15,624::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0ed7114-6873-49bc-90ec-3d515963cfe3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420263', 'lastCheck': '2.8', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:15,624::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:15:15,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0ed7114-6873-49bc-90ec-3d515963cfe3`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:15,626::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:15,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:15:15,629::task::993::Storage.TaskManager.Task::(_decref) Task=`d0ed7114-6873-49bc-90ec-3d515963cfe3`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:15:15,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 15:15:16,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`b44f142b-1b25-4bda-a5e3-bb18c1ac4a69`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:15:16,688::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:15:16,688::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:16,689::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:16,689::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:16,689::__init__::232::IOProcess::(_processLogs) (6630) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:15:16,690::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6630) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:16,691::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:16,692::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:16,692::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:16,692::__init__::232::IOProcess::(_processLogs) (6631) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:15:16,693::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6631) Queuing response >Thread-55::INFO::2014-08-11 15:15:16,693::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:15:16,694::task::1191::Storage.TaskManager.Task::(prepare) Task=`b44f142b-1b25-4bda-a5e3-bb18c1ac4a69`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:15:16,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`b44f142b-1b25-4bda-a5e3-bb18c1ac4a69`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:15:16,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:15:16,695::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:15:16,695::task::993::Storage.TaskManager.Task::(_decref) Task=`b44f142b-1b25-4bda-a5e3-bb18c1ac4a69`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:15:17,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:17,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133082 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:19,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:19,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132722 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:15:20,580::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:15:20,581::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e01e735-4a95-4768-85a1-b3c35caefd1d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:15:20,619::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:15:20,633::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:20,635::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e01e735-4a95-4768-85a1-b3c35caefd1d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:20,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:15:20,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e01e735-4a95-4768-85a1-b3c35caefd1d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:20,638::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:20,639::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:15:20,640::task::993::Storage.TaskManager.Task::(_decref) Task=`9e01e735-4a95-4768-85a1-b3c35caefd1d`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:20,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:20,644::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:15:20,655::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:15:20,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdb72fac-5ff3-4ec0-b525-ae3e047d1a7b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:15:20,656::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:15:20,657::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ff9bf961-bcaa-41e2-b891-6bfb79123ef0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:15:20,657::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:15:20,657::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:15:20,658::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ff9bf961-bcaa-41e2-b891-6bfb79123ef0`::Granted request >Thread-34::DEBUG::2014-08-11 15:15:20,658::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bdb72fac-5ff3-4ec0-b525-ae3e047d1a7b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:15:20,658::task::993::Storage.TaskManager.Task::(_decref) Task=`bdb72fac-5ff3-4ec0-b525-ae3e047d1a7b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:15:20,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856980975616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:20,679::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdb72fac-5ff3-4ec0-b525-ae3e047d1a7b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856980975616', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:20,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdb72fac-5ff3-4ec0-b525-ae3e047d1a7b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:15:20,679::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:15:20,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:15:20,680::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:15:20,680::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:15:20,680::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:15:20,680::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:15:20,681::task::993::Storage.TaskManager.Task::(_decref) Task=`bdb72fac-5ff3-4ec0-b525-ae3e047d1a7b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:15:21,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:21,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013483 s, 75.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:22,818::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:22,818::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:22,819::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:22,819::__init__::232::IOProcess::(_processLogs) (6632) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:22,820::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6632) Queuing response >Thread-16::DEBUG::2014-08-11 15:15:22,821::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:15:22,841::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000409573 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:22,842::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:22,842::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:22,843::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:22,843::__init__::232::IOProcess::(_processLogs) (6633) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:22,843::__init__::232::IOProcess::(_processLogs) (6633) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:22,844::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:22,844::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:22,845::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:22,845::__init__::232::IOProcess::(_processLogs) (6634) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:22,846::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6634) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:22,846::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:22,847::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:22,848::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:22,848::__init__::232::IOProcess::(_processLogs) (6635) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:22,848::__init__::232::IOProcess::(_processLogs) (6635) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:22,848::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:22,849::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:22,849::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:22,849::__init__::232::IOProcess::(_processLogs) (6636) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:22,850::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6636) Queuing response >Dummy-33::DEBUG::2014-08-11 15:15:23,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:23,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130986 s, 78.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:25,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:25,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132042 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:25,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:25,652::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:25,720::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:15:27,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:27,587::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0128779 s, 79.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:29,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:29,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013748 s, 74.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:15:30,693::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:15:30,694::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ee48520-f4dc-4048-ae20-a528749e04d7`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:15:30,719::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:30,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:30,725::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:30,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-113::DEBUG::2014-08-11 15:15:30,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`057670d4-159e-4949-a55a-5cfe97d3a8e5`::moving from state init -> state preparing >Thread-113::INFO::2014-08-11 15:15:30,743::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-113::INFO::2014-08-11 15:15:30,744::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000409573', 'lastCheck': '7.9', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:15:30,744::task::1191::Storage.TaskManager.Task::(prepare) Task=`057670d4-159e-4949-a55a-5cfe97d3a8e5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000409573', 'lastCheck': '7.9', 'valid': True}} >Thread-113::DEBUG::2014-08-11 15:15:30,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`057670d4-159e-4949-a55a-5cfe97d3a8e5`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-08-11 15:15:30,744::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-08-11 15:15:30,745::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-08-11 15:15:30,745::task::993::Storage.TaskManager.Task::(_decref) Task=`057670d4-159e-4949-a55a-5cfe97d3a8e5`::ref 0 aborting False >Thread-113::DEBUG::2014-08-11 15:15:30,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:15:30,770::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:30,771::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ee48520-f4dc-4048-ae20-a528749e04d7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:30,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ee48520-f4dc-4048-ae20-a528749e04d7`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:30,774::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:30,774::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:15:30,774::task::993::Storage.TaskManager.Task::(_decref) Task=`0ee48520-f4dc-4048-ae20-a528749e04d7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:15:30,789::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:15:30,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`07b0f2ea-f673-4c17-90af-02dc31f4dd1d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:15:30,790::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:15:30,791::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e18ec9a7-7caf-4cbf-82d4-b358c89ddde6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:15:30,791::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:15:30,792::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:15:30,792::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e18ec9a7-7caf-4cbf-82d4-b358c89ddde6`::Granted request >Thread-34::DEBUG::2014-08-11 15:15:30,793::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`07b0f2ea-f673-4c17-90af-02dc31f4dd1d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:15:30,793::task::993::Storage.TaskManager.Task::(_decref) Task=`07b0f2ea-f673-4c17-90af-02dc31f4dd1d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:15:30,813::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856814514176', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:30,813::task::1191::Storage.TaskManager.Task::(prepare) Task=`07b0f2ea-f673-4c17-90af-02dc31f4dd1d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856814514176', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:30,813::task::595::Storage.TaskManager.Task::(_updateState) Task=`07b0f2ea-f673-4c17-90af-02dc31f4dd1d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:15:30,813::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:15:30,814::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:15:30,814::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:15:30,814::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:15:30,815::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:15:30,815::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:15:30,815::task::993::Storage.TaskManager.Task::(_decref) Task=`07b0f2ea-f673-4c17-90af-02dc31f4dd1d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:15:31,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:31,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133969 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:32,853::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:32,854::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:32,855::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:32,856::__init__::232::IOProcess::(_processLogs) (6637) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:32,858::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6637) Queuing response >Thread-16::DEBUG::2014-08-11 15:15:32,860::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:15:32,880::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000539651 s, 843 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:32,883::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:32,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:32,887::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:32,888::__init__::232::IOProcess::(_processLogs) (6638) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:32,891::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6638) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:32,894::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:32,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:32,897::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:32,898::__init__::232::IOProcess::(_processLogs) (6639) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:32,900::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6639) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:32,903::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:32,904::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:32,906::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:32,906::__init__::232::IOProcess::(_processLogs) (6640) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:32,908::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6640) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:32,911::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:32,913::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:32,914::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:32,916::__init__::232::IOProcess::(_processLogs) (6641) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:32,917::__init__::232::IOProcess::(_processLogs) (6641) Queuing response >Dummy-33::DEBUG::2014-08-11 15:15:33,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:33,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132152 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:35,733::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:35,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:15:35,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:35,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:15:35,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133883 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:37,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:37,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132398 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:39,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:39,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132736 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:15:40,827::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:15:40,828::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d9f2bae-ebe7-45f6-9ee8-87c7cbaa6714`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:15:40,829::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:15:40,841::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:40,842::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d9f2bae-ebe7-45f6-9ee8-87c7cbaa6714`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:40,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d9f2bae-ebe7-45f6-9ee8-87c7cbaa6714`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:40,842::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:40,842::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:15:40,842::task::993::Storage.TaskManager.Task::(_decref) Task=`3d9f2bae-ebe7-45f6-9ee8-87c7cbaa6714`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:40,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:40,848::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:40,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:15:40,854::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:15:40,855::task::595::Storage.TaskManager.Task::(_updateState) Task=`e03f0711-1db8-44ff-aadf-7dbbfabfaf93`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:15:40,855::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:15:40,856::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6161cb2e-280e-4597-9037-41a9792493c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:15:40,856::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:15:40,857::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:15:40,857::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6161cb2e-280e-4597-9037-41a9792493c4`::Granted request >Thread-34::DEBUG::2014-08-11 15:15:40,857::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e03f0711-1db8-44ff-aadf-7dbbfabfaf93`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:15:40,858::task::993::Storage.TaskManager.Task::(_decref) Task=`e03f0711-1db8-44ff-aadf-7dbbfabfaf93`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:15:40,878::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856649101312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:40,878::task::1191::Storage.TaskManager.Task::(prepare) Task=`e03f0711-1db8-44ff-aadf-7dbbfabfaf93`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856649101312', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:40,878::task::595::Storage.TaskManager.Task::(_updateState) Task=`e03f0711-1db8-44ff-aadf-7dbbfabfaf93`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:15:40,879::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:15:40,879::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:15:40,879::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:15:40,879::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:15:40,880::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:15:40,880::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:15:40,880::task::993::Storage.TaskManager.Task::(_decref) Task=`e03f0711-1db8-44ff-aadf-7dbbfabfaf93`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:15:41,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:42,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132297 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:42,913::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:42,914::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:42,914::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:42,914::__init__::232::IOProcess::(_processLogs) (6642) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:42,914::__init__::232::IOProcess::(_processLogs) (6642) Queuing response >Thread-16::DEBUG::2014-08-11 15:15:42,915::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:15:42,934::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041579 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:42,936::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:42,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:42,936::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:42,936::__init__::232::IOProcess::(_processLogs) (6643) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:42,937::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6643) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:42,939::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:42,939::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:42,939::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:42,940::__init__::232::IOProcess::(_processLogs) (6644) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:42,940::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6644) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:42,941::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:42,941::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:42,942::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:42,942::__init__::232::IOProcess::(_processLogs) (6645) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:42,942::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6645) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:42,943::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:42,944::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:42,944::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:42,944::__init__::232::IOProcess::(_processLogs) (6646) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:42,945::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6646) Queuing response >Dummy-33::DEBUG::2014-08-11 15:15:44,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:44,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132351 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:45,856::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:45,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:15:45,871::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d37d2ff-5f88-4b76-b974-dbb7966d8ba5`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:15:45,920::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:15:45,922::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041579', 'lastCheck': '3.0', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:45,924::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:15:45,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d37d2ff-5f88-4b76-b974-dbb7966d8ba5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041579', 'lastCheck': '3.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:15:45,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d37d2ff-5f88-4b76-b974-dbb7966d8ba5`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:45,927::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:45,928::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:15:45,928::task::993::Storage.TaskManager.Task::(_decref) Task=`6d37d2ff-5f88-4b76-b974-dbb7966d8ba5`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:15:45,947::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:15:46,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:46,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133938 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:48,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:48,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136468 s, 75.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:50,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:50,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133573 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:15:50,893::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:15:50,894::task::595::Storage.TaskManager.Task::(_updateState) Task=`b63e5bf3-1a91-4ba9-9e9d-b5ad0439a611`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:15:50,919::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:15:50,932::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:50,933::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:15:50,934::task::1191::Storage.TaskManager.Task::(prepare) Task=`b63e5bf3-1a91-4ba9-9e9d-b5ad0439a611`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:15:50,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`b63e5bf3-1a91-4ba9-9e9d-b5ad0439a611`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:15:50,937::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:15:50,938::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:50,938::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:15:50,939::task::993::Storage.TaskManager.Task::(_decref) Task=`b63e5bf3-1a91-4ba9-9e9d-b5ad0439a611`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:50,943::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:15:50,954::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:15:50,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad5d91ed-9891-4e8b-bb81-2fcd802f6aa9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:15:50,954::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:15:50,955::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d7f46d9-aad4-4937-904e-0a05e7a44340`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:15:50,955::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:15:50,956::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:15:50,956::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d7f46d9-aad4-4937-904e-0a05e7a44340`::Granted request >Thread-34::DEBUG::2014-08-11 15:15:50,957::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ad5d91ed-9891-4e8b-bb81-2fcd802f6aa9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:15:50,957::task::993::Storage.TaskManager.Task::(_decref) Task=`ad5d91ed-9891-4e8b-bb81-2fcd802f6aa9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:15:50,977::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856514162688', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:50,977::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad5d91ed-9891-4e8b-bb81-2fcd802f6aa9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856514162688', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:15:50,977::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad5d91ed-9891-4e8b-bb81-2fcd802f6aa9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:15:50,978::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:15:50,978::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:15:50,978::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:15:50,979::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:15:50,979::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:15:50,979::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:15:50,979::task::993::Storage.TaskManager.Task::(_decref) Task=`ad5d91ed-9891-4e8b-bb81-2fcd802f6aa9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:15:52,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:52,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212519 s, 48.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:52,947::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:52,947::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:52,947::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:52,948::__init__::232::IOProcess::(_processLogs) (6647) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:52,948::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6647) Queuing response >Thread-16::DEBUG::2014-08-11 15:15:52,949::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:15:52,967::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432038 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:15:52,968::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:52,969::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:52,969::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:52,969::__init__::232::IOProcess::(_processLogs) (6648) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:15:52,970::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6648) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:52,972::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:52,972::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:52,972::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:52,973::__init__::232::IOProcess::(_processLogs) (6649) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:52,973::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6649) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:52,974::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:52,974::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:52,974::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:52,974::__init__::232::IOProcess::(_processLogs) (6650) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:52,975::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6650) Queuing response >Thread-14::DEBUG::2014-08-11 15:15:52,976::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:15:52,976::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:15:52,976::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:15:52,977::__init__::232::IOProcess::(_processLogs) (6651) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:15:52,977::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6651) Queuing response >Dummy-33::DEBUG::2014-08-11 15:15:54,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:54,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132504 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:55,948::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:55,949::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:15:56,020::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:15:56,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:56,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136561 s, 75.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:15:58,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:15:58,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133498 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:00,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:00,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132977 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:16:00,992::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:16:00,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a29d6d6-1e02-477f-a9c9-166059fc6acf`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:00,994::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:16:01,015::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:01,015::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a29d6d6-1e02-477f-a9c9-166059fc6acf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:01,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a29d6d6-1e02-477f-a9c9-166059fc6acf`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:01,015::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:01,016::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:01,016::task::993::Storage.TaskManager.Task::(_decref) Task=`4a29d6d6-1e02-477f-a9c9-166059fc6acf`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:16:01,027::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:01,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:16:01,030::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea69a954-3774-4a8a-9f79-2979ef89ff93`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:16:01,032::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:01,036::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:16:01,036::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fa3f8ab8-caf2-4c33-84c5-a0947f48fc29`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:16:01,037::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:16:01,040::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:01,041::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:16:01,041::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fa3f8ab8-caf2-4c33-84c5-a0947f48fc29`::Granted request >Thread-34::DEBUG::2014-08-11 15:16:01,044::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ea69a954-3774-4a8a-9f79-2979ef89ff93`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:16:01,046::task::993::Storage.TaskManager.Task::(_decref) Task=`ea69a954-3774-4a8a-9f79-2979ef89ff93`::ref 1 aborting False >Thread-115::DEBUG::2014-08-11 15:16:01,050::task::595::Storage.TaskManager.Task::(_updateState) Task=`3496ce12-823a-450a-ab43-32634f17cbdb`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:01,052::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:16:01,055::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432038', 'lastCheck': '8.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:16:01,057::task::1191::Storage.TaskManager.Task::(prepare) Task=`3496ce12-823a-450a-ab43-32634f17cbdb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432038', 'lastCheck': '8.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:16:01,058::task::595::Storage.TaskManager.Task::(_updateState) Task=`3496ce12-823a-450a-ab43-32634f17cbdb`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:01,059::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:01,060::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:01,063::task::993::Storage.TaskManager.Task::(_decref) Task=`3496ce12-823a-450a-ab43-32634f17cbdb`::ref 0 aborting False >Thread-34::INFO::2014-08-11 15:16:01,099::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856387743744', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:01,099::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea69a954-3774-4a8a-9f79-2979ef89ff93`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856387743744', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:01,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea69a954-3774-4a8a-9f79-2979ef89ff93`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:16:01,100::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:16:01,100::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:16:01,100::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:16:01,101::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:16:01,101::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:16:01,101::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:16:01,102::task::993::Storage.TaskManager.Task::(_decref) Task=`ea69a954-3774-4a8a-9f79-2979ef89ff93`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:16:01,104::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:16:02,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:02,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283032 s, 36.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:02,979::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:02,979::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:02,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:02,980::__init__::232::IOProcess::(_processLogs) (6652) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:02,980::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|(6652) Queuing response >Thread-16::DEBUG::2014-08-11 15:16:02,981::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:16:03,000::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423774 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:03,001::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:03,002::__init__::232::IOProcess::(_processLogs) (6596) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:03,002::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:03,002::__init__::232::IOProcess::(_processLogs) (6653) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:03,003::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:03,004::__init__::232::IOProcess::(_processLogs) (6654) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:03,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:03,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:03,006::__init__::232::IOProcess::(_processLogs) (6655) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:03,007::__init__::232::IOProcess::(_processLogs) (6655) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:03,007::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:03,007::__init__::232::IOProcess::(_processLogs) (6656) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:03,008::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6656) Queuing response >Dummy-33::DEBUG::2014-08-11 15:16:04,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:04,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136417 s, 75.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:06,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:06,052::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:06,121::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:16:06,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:06,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132997 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:08,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:09,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132702 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:11,031::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:11,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131493 s, 77.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:16:11,120::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:16:11,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e7b3105-719f-43dc-88c8-3806f6280a6e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:11,122::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:11,128::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:11,129::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:11,130::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:16:11,135::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:11,135::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e7b3105-719f-43dc-88c8-3806f6280a6e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:11,135::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e7b3105-719f-43dc-88c8-3806f6280a6e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:11,135::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:11,136::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:11,136::task::993::Storage.TaskManager.Task::(_decref) Task=`2e7b3105-719f-43dc-88c8-3806f6280a6e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:16:11,146::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:16:11,147::task::595::Storage.TaskManager.Task::(_updateState) Task=`43abf44b-de52-42bf-8a2f-e8209c272b51`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:16:11,147::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:16:11,148::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b85e4b3e-0be6-4715-8ff2-c1282ca7bf1b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:16:11,148::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:16:11,148::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:16:11,148::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b85e4b3e-0be6-4715-8ff2-c1282ca7bf1b`::Granted request >Thread-34::DEBUG::2014-08-11 15:16:11,149::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`43abf44b-de52-42bf-8a2f-e8209c272b51`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:16:11,149::task::993::Storage.TaskManager.Task::(_decref) Task=`43abf44b-de52-42bf-8a2f-e8209c272b51`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:16:11,169::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856260472832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:11,169::task::1191::Storage.TaskManager.Task::(prepare) Task=`43abf44b-de52-42bf-8a2f-e8209c272b51`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856260472832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:11,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`43abf44b-de52-42bf-8a2f-e8209c272b51`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:16:11,170::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:16:11,170::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:16:11,170::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:16:11,171::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:16:11,171::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:16:11,171::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:16:11,171::task::993::Storage.TaskManager.Task::(_decref) Task=`43abf44b-de52-42bf-8a2f-e8209c272b51`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:16:13,010::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:13,010::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:13,010::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:13,011::__init__::232::IOProcess::(_processLogs) (6657) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:13,011::__init__::232::IOProcess::(_processLogs) (6657) Queuing response >Thread-16::DEBUG::2014-08-11 15:16:13,012::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:16:13,035::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00178892 s, 254 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:13,036::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:13,036::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:13,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:13,037::__init__::232::IOProcess::(_processLogs) (6658) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:13,038::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:13,038::__init__::232::IOProcess::(_processLogs) (6659) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:13,039::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6659) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:13,039::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:13,040::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:13,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:13,040::__init__::232::IOProcess::(_processLogs) (6660) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:13,041::__init__::232::IOProcess::(_processLogs) (6660) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:13,041::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:13,042::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:13,042::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:13,042::__init__::232::IOProcess::(_processLogs) (6661) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:13,042::__init__::232::IOProcess::(_processLogs) (6661) Queuing response >Dummy-33::DEBUG::2014-08-11 15:16:13,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:13,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133939 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:15,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:15,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132735 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:16,137::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:16,139::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:16:16,198::task::595::Storage.TaskManager.Task::(_updateState) Task=`c387a2fc-bfa1-41fa-966d-228532ab6201`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:16,222::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:16:16,224::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00178892', 'lastCheck': '3.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:16,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:16:16,225::task::1191::Storage.TaskManager.Task::(prepare) Task=`c387a2fc-bfa1-41fa-966d-228532ab6201`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00178892', 'lastCheck': '3.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:16:16,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`c387a2fc-bfa1-41fa-966d-228532ab6201`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:16,228::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:16,229::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:16,229::task::993::Storage.TaskManager.Task::(_decref) Task=`c387a2fc-bfa1-41fa-966d-228532ab6201`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:16:16,250::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 15:16:16,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`efc8c483-173a-400b-bf18-9efd3c698676`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:16:16,804::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:16:16,804::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:16,805::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:16,805::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:16,805::__init__::232::IOProcess::(_processLogs) (6662) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:16:16,806::__init__::232::IOProcess::(_processLogs) DEBUG|(6662) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:16,807::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:16,807::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 15:16:16,808::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:16:16,808::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:16:16,809::task::1191::Storage.TaskManager.Task::(prepare) Task=`efc8c483-173a-400b-bf18-9efd3c698676`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:16:16,810::__init__::232::IOProcess::(_processLogs) (6663) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:16:16,811::task::595::Storage.TaskManager.Task::(_updateState) Task=`efc8c483-173a-400b-bf18-9efd3c698676`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:16:16,811::__init__::232::IOProcess::(_processLogs) (6663) Queuing response >Thread-55::DEBUG::2014-08-11 15:16:16,812::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:16:16,813::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:16:16,813::task::993::Storage.TaskManager.Task::(_decref) Task=`efc8c483-173a-400b-bf18-9efd3c698676`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:16:17,198::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:17,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132826 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:19,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:19,286::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131988 s, 77.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:16:21,184::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:16:21,185::task::595::Storage.TaskManager.Task::(_updateState) Task=`58d7b18e-458e-46c3-9bc4-7b7d11442972`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:21,220::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:16:21,234::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:21,236::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:16:21,236::task::1191::Storage.TaskManager.Task::(prepare) Task=`58d7b18e-458e-46c3-9bc4-7b7d11442972`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:21,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`58d7b18e-458e-46c3-9bc4-7b7d11442972`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:21,238::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:21,240::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:21,241::task::993::Storage.TaskManager.Task::(_decref) Task=`58d7b18e-458e-46c3-9bc4-7b7d11442972`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:21,241::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:21,245::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:16:21,254::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:16:21,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`1853d250-b31c-4156-a907-aab3d90be6a3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:16:21,255::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:16:21,256::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5116da4b-1627-4852-a351-04b03c962a1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:16:21,256::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:16:21,257::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:16:21,257::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5116da4b-1627-4852-a351-04b03c962a1d`::Granted request >Thread-34::DEBUG::2014-08-11 15:16:21,257::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1853d250-b31c-4156-a907-aab3d90be6a3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:16:21,258::task::993::Storage.TaskManager.Task::(_decref) Task=`1853d250-b31c-4156-a907-aab3d90be6a3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:16:21,277::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856090734592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:21,278::task::1191::Storage.TaskManager.Task::(prepare) Task=`1853d250-b31c-4156-a907-aab3d90be6a3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1856090734592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:21,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`1853d250-b31c-4156-a907-aab3d90be6a3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:16:21,278::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:16:21,278::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:16:21,279::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:16:21,279::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:16:21,279::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:16:21,279::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:16:21,280::task::993::Storage.TaskManager.Task::(_decref) Task=`1853d250-b31c-4156-a907-aab3d90be6a3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:16:21,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:21,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139212 s, 73.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:23,044::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:23,044::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:23,045::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:16:23,047::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:16:23,047::__init__::232::IOProcess::(_processLogs) (6664) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:23,065::__init__::232::IOProcess::(_processLogs) (6664) Queuing response >Thread-16::DEBUG::2014-08-11 15:16:23,067::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426856 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:23,068::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:23,069::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:23,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:23,069::__init__::232::IOProcess::(_processLogs) (6665) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:23,071::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:23,071::__init__::232::IOProcess::(_processLogs) (6666) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:23,072::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:23,072::__init__::232::IOProcess::(_processLogs) (6667) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:23,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:23,073::__init__::232::IOProcess::(_processLogs) (6668) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:16:23,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:23,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130831 s, 78.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:25,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:25,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131872 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:26,248::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:26,249::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:26,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:16:27,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:27,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134383 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:29,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:29,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136579 s, 75.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:16:31,292::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:16:31,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bb4b405-3708-4081-a0ef-4775dae5108a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:31,320::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:31,328::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:31,331::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:31,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:16:31,333::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:31,334::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bb4b405-3708-4081-a0ef-4775dae5108a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:31,334::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bb4b405-3708-4081-a0ef-4775dae5108a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:31,334::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:31,334::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:31,335::task::993::Storage.TaskManager.Task::(_decref) Task=`8bb4b405-3708-4081-a0ef-4775dae5108a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:16:31,346::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:16:31,347::task::595::Storage.TaskManager.Task::(_updateState) Task=`a62dd81c-b768-4bc9-9e16-635ae2287d14`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:16:31,347::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:16:31,348::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fc3873fe-c361-40e9-b0f1-5bbeea2261bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:16:31,348::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:16:31,348::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:16:31,348::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fc3873fe-c361-40e9-b0f1-5bbeea2261bd`::Granted request >Thread-34::DEBUG::2014-08-11 15:16:31,349::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a62dd81c-b768-4bc9-9e16-635ae2287d14`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:16:31,349::task::993::Storage.TaskManager.Task::(_decref) Task=`a62dd81c-b768-4bc9-9e16-635ae2287d14`::ref 1 aborting False >Thread-115::DEBUG::2014-08-11 15:16:31,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`a345244e-ea2a-4d3c-8755-f74ff5d34577`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:31,358::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:16:31,361::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426856', 'lastCheck': '8.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:16:31,363::task::1191::Storage.TaskManager.Task::(prepare) Task=`a345244e-ea2a-4d3c-8755-f74ff5d34577`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426856', 'lastCheck': '8.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:16:31,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`a345244e-ea2a-4d3c-8755-f74ff5d34577`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:31,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:31,367::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:31,369::task::993::Storage.TaskManager.Task::(_decref) Task=`a345244e-ea2a-4d3c-8755-f74ff5d34577`::ref 0 aborting False >Thread-34::INFO::2014-08-11 15:16:31,402::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855928270848', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:31,402::task::1191::Storage.TaskManager.Task::(prepare) Task=`a62dd81c-b768-4bc9-9e16-635ae2287d14`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855928270848', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:31,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`a62dd81c-b768-4bc9-9e16-635ae2287d14`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:16:31,402::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:16:31,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:16:31,403::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:16:31,403::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:16:31,404::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:16:31,404::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:16:31,404::task::993::Storage.TaskManager.Task::(_decref) Task=`a62dd81c-b768-4bc9-9e16-635ae2287d14`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:16:31,409::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:16:31,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:31,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135146 s, 75.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:33,075::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:33,076::__init__::232::IOProcess::(_processLogs) (6669) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:16:33,076::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:16:33,095::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0004443 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:33,096::__init__::232::IOProcess::(_processLogs) (6670) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:33,098::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:33,098::__init__::232::IOProcess::(_processLogs) (6671) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:33,099::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:33,099::__init__::232::IOProcess::(_processLogs) (6672) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:33,100::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:33,100::__init__::232::IOProcess::(_processLogs) (6673) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:33,101::__init__::232::IOProcess::(_processLogs) (6673) Queuing response >Dummy-33::DEBUG::2014-08-11 15:16:33,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:33,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131618 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:35,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:35,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131306 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:36,339::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:36,341::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:36,421::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:16:37,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:37,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131833 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:39,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:39,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133001 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:41,423::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:41,424::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:16:41,427::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:16:41,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc05d8c7-fb4e-459d-ad0f-716583952b1a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:41,524::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:41,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:16:41,539::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:41,539::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc05d8c7-fb4e-459d-ad0f-716583952b1a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:41,539::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc05d8c7-fb4e-459d-ad0f-716583952b1a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:41,540::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:41,540::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:41,540::task::993::Storage.TaskManager.Task::(_decref) Task=`dc05d8c7-fb4e-459d-ad0f-716583952b1a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:16:41,553::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:16:41,554::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0b8b279-464c-44b4-8972-59b7f767c6b1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:16:41,554::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:16:41,555::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6eb210ef-0aca-4a8e-bc71-20b147ac01c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:16:41,555::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:16:41,556::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:16:41,556::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6eb210ef-0aca-4a8e-bc71-20b147ac01c6`::Granted request >Thread-34::DEBUG::2014-08-11 15:16:41,556::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a0b8b279-464c-44b4-8972-59b7f767c6b1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:16:41,557::task::993::Storage.TaskManager.Task::(_decref) Task=`a0b8b279-464c-44b4-8972-59b7f767c6b1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:16:41,576::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855799558144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:41,577::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0b8b279-464c-44b4-8972-59b7f767c6b1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855799558144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:41,577::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0b8b279-464c-44b4-8972-59b7f767c6b1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:16:41,577::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:16:41,577::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:16:41,578::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:16:41,578::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:16:41,578::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:16:41,579::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:16:41,579::task::993::Storage.TaskManager.Task::(_decref) Task=`a0b8b279-464c-44b4-8972-59b7f767c6b1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:16:41,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:41,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133344 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:43,103::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:43,103::__init__::232::IOProcess::(_processLogs) (6674) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:16:43,104::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:16:43,123::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000429234 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:43,124::__init__::232::IOProcess::(_processLogs) (6675) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:43,126::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:43,126::__init__::232::IOProcess::(_processLogs) (6676) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:43,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:43,127::__init__::232::IOProcess::(_processLogs) (6677) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:43,128::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6677) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:43,129::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:43,130::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:43,131::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:43,131::__init__::232::IOProcess::(_processLogs) (6678) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:43,131::__init__::232::IOProcess::(_processLogs) (6678) Queuing response >Dummy-33::DEBUG::2014-08-11 15:16:43,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:43,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134899 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:45,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:46,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132567 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:16:46,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0951a4c-5afd-4baf-bc56-4447b267bc02`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:16:46,498::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:16:46,499::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429234', 'lastCheck': '3.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:16:46,499::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0951a4c-5afd-4baf-bc56-4447b267bc02`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429234', 'lastCheck': '3.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:16:46,499::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0951a4c-5afd-4baf-bc56-4447b267bc02`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:46,499::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:46,500::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:46,500::task::993::Storage.TaskManager.Task::(_decref) Task=`a0951a4c-5afd-4baf-bc56-4447b267bc02`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:16:46,519::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:46,530::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:46,531::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:46,532::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:16:48,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:48,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136608 s, 75.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:50,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:50,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132838 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:51,535::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:51,536::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:16:51,591::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:16:51,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f816a43-3478-4d41-adcb-1835c619cbfb`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:51,625::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:16:51,625::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:16:51,639::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:51,640::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f816a43-3478-4d41-adcb-1835c619cbfb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:16:51,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f816a43-3478-4d41-adcb-1835c619cbfb`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:16:51,642::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:16:51,642::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:16:51,643::task::993::Storage.TaskManager.Task::(_decref) Task=`1f816a43-3478-4d41-adcb-1835c619cbfb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:16:51,655::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:16:51,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`15618a54-8134-4ea5-a532-2d3c0f0689c8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:16:51,656::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:16:51,657::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`034ce106-59e9-486d-a91a-141fea880156`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:16:51,657::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:16:51,657::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:16:51,657::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`034ce106-59e9-486d-a91a-141fea880156`::Granted request >Thread-34::DEBUG::2014-08-11 15:16:51,658::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`15618a54-8134-4ea5-a532-2d3c0f0689c8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:16:51,658::task::993::Storage.TaskManager.Task::(_decref) Task=`15618a54-8134-4ea5-a532-2d3c0f0689c8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:16:51,678::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855665012736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:51,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`15618a54-8134-4ea5-a532-2d3c0f0689c8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855665012736', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:16:51,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`15618a54-8134-4ea5-a532-2d3c0f0689c8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:16:51,679::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:16:51,679::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:16:51,679::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:16:51,680::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:16:51,680::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:16:51,680::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:16:51,680::task::993::Storage.TaskManager.Task::(_decref) Task=`15618a54-8134-4ea5-a532-2d3c0f0689c8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:16:52,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:52,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132419 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:53,132::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:53,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:16:53,134::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:16:53,134::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:53,151::__init__::232::IOProcess::(_processLogs) (6679) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:53,152::__init__::232::IOProcess::(_processLogs) (6679) Queuing response >Thread-16::DEBUG::2014-08-11 15:16:53,153::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00091745 s, 496 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:16:53,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:53,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:53,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:53,155::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6680) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:16:53,155::__init__::232::IOProcess::(_processLogs) (6680) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:53,156::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:53,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:53,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:53,157::__init__::232::IOProcess::(_processLogs) (6681) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:53,158::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6681) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:53,159::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:53,159::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:53,159::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:53,160::__init__::232::IOProcess::(_processLogs) (6682) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:53,160::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6682) Queuing response >Thread-14::DEBUG::2014-08-11 15:16:53,161::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:16:53,161::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:16:53,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:16:53,162::__init__::232::IOProcess::(_processLogs) (6683) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:16:53,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6683) Queuing response >Dummy-33::DEBUG::2014-08-11 15:16:54,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:54,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133321 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:16:56,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:56,318::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132608 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:56,632::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:56,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:16:56,721::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:16:58,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:16:58,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131794 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:00,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:00,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0236838 s, 43.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:17:01,606::task::595::Storage.TaskManager.Task::(_updateState) Task=`778243a9-d90f-47b4-b9f8-6a04e54ec777`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:01,606::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:17:01,607::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00091745', 'lastCheck': '8.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:01,607::task::1191::Storage.TaskManager.Task::(prepare) Task=`778243a9-d90f-47b4-b9f8-6a04e54ec777`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00091745', 'lastCheck': '8.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:01,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`778243a9-d90f-47b4-b9f8-6a04e54ec777`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:01,608::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:01,608::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:01,608::task::993::Storage.TaskManager.Task::(_decref) Task=`778243a9-d90f-47b4-b9f8-6a04e54ec777`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:17:01,630::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:17:01,692::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:17:01,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`1737667e-b2d5-4e2f-bd79-72f068bcf8b3`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:01,720::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:01,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:01,730::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:01,731::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:17:01,733::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:01,734::task::1191::Storage.TaskManager.Task::(prepare) Task=`1737667e-b2d5-4e2f-bd79-72f068bcf8b3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:01,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`1737667e-b2d5-4e2f-bd79-72f068bcf8b3`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:01,734::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:01,734::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:01,735::task::993::Storage.TaskManager.Task::(_decref) Task=`1737667e-b2d5-4e2f-bd79-72f068bcf8b3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:17:01,749::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:17:01,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`d163eab8-06fb-4037-b907-626541e2235c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:17:01,750::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:17:01,751::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b1c63820-d04b-4b85-9738-8a0ff11c6cfc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:17:01,751::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:17:01,751::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:17:01,751::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b1c63820-d04b-4b85-9738-8a0ff11c6cfc`::Granted request >Thread-34::DEBUG::2014-08-11 15:17:01,752::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d163eab8-06fb-4037-b907-626541e2235c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:17:01,752::task::993::Storage.TaskManager.Task::(_decref) Task=`d163eab8-06fb-4037-b907-626541e2235c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:17:01,772::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855590694912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:01,773::task::1191::Storage.TaskManager.Task::(prepare) Task=`d163eab8-06fb-4037-b907-626541e2235c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855590694912', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:01,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`d163eab8-06fb-4037-b907-626541e2235c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:17:01,773::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:17:01,773::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:17:01,774::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:17:01,774::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:17:01,774::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:17:01,775::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:17:01,775::task::993::Storage.TaskManager.Task::(_decref) Task=`d163eab8-06fb-4037-b907-626541e2235c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:17:02,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:02,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271698 s, 37.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:03,164::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:03,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:03,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:03,166::__init__::232::IOProcess::(_processLogs) (6684) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:03,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(6684) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:03,167::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:17:03,185::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000435327 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:03,186::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:03,187::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:03,187::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:03,187::__init__::232::IOProcess::(_processLogs) (6685) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:03,189::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:03,189::__init__::232::IOProcess::(_processLogs) (6686) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:03,189::__init__::232::IOProcess::(_processLogs) DEBUG|(6686) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:03,190::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:03,190::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:03,191::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:03,191::__init__::232::IOProcess::(_processLogs) DEBUG|(6687) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:03,191::__init__::232::IOProcess::(_processLogs) (6687) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:03,192::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:03,192::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:03,193::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:03,193::__init__::232::IOProcess::(_processLogs) DEBUG|(6688) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:03,193::__init__::232::IOProcess::(_processLogs) (6688) Queuing response >Dummy-33::DEBUG::2014-08-11 15:17:04,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:04,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0211916 s, 48.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:06,738::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:06,739::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:06,821::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:17:06,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:06,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013363 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:08,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:08,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132206 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:10,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:10,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013422 s, 76.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:17:11,787::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:17:11,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc3e7f2a-4b2c-4266-817a-cdd31c81c2df`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:11,820::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:11,828::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:11,830::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:11,833::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:17:11,834::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:11,835::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc3e7f2a-4b2c-4266-817a-cdd31c81c2df`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:11,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc3e7f2a-4b2c-4266-817a-cdd31c81c2df`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:11,835::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:11,835::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:11,836::task::993::Storage.TaskManager.Task::(_decref) Task=`fc3e7f2a-4b2c-4266-817a-cdd31c81c2df`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:17:11,846::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:17:11,846::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf14aba0-7b42-43f0-ade8-6b6074f3a4c3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:17:11,847::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:17:11,847::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`031c392b-6899-4d77-bac4-709d325342bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:17:11,848::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:17:11,848::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:17:11,848::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`031c392b-6899-4d77-bac4-709d325342bd`::Granted request >Thread-34::DEBUG::2014-08-11 15:17:11,849::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bf14aba0-7b42-43f0-ade8-6b6074f3a4c3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:17:11,849::task::993::Storage.TaskManager.Task::(_decref) Task=`bf14aba0-7b42-43f0-ade8-6b6074f3a4c3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:17:11,869::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855569002496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:11,869::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf14aba0-7b42-43f0-ade8-6b6074f3a4c3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855569002496', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:11,869::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf14aba0-7b42-43f0-ade8-6b6074f3a4c3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:17:11,870::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:17:11,870::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:17:11,870::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:17:11,870::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:17:11,871::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:17:11,871::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:17:11,871::task::993::Storage.TaskManager.Task::(_decref) Task=`bf14aba0-7b42-43f0-ade8-6b6074f3a4c3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:17:13,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:13,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133462 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:13,195::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:13,196::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:13,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:13,196::__init__::232::IOProcess::(_processLogs) (6689) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:13,197::__init__::232::IOProcess::(_processLogs) (6689) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:13,198::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:17:13,217::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416409 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:13,218::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:13,218::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:13,219::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:13,219::__init__::232::IOProcess::(_processLogs) (6690) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:13,220::__init__::232::IOProcess::(_processLogs) DEBUG|(6690) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:13,221::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:13,222::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:13,222::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:13,223::__init__::232::IOProcess::(_processLogs) (6691) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:13,223::__init__::232::IOProcess::(_processLogs) (6691) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:13,223::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:13,224::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:13,225::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:13,225::__init__::232::IOProcess::(_processLogs) (6692) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:13,225::__init__::232::IOProcess::(_processLogs) (6692) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:13,225::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:13,226::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:13,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:13,227::__init__::232::IOProcess::(_processLogs) (6693) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:13,228::__init__::232::IOProcess::(_processLogs) (6693) Queuing response >Dummy-33::DEBUG::2014-08-11 15:17:15,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:15,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013264 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:17:16,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9c3b180-a37d-4b41-96c7-ff1f0204c3ff`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:16,740::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:17:16,740::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416409', 'lastCheck': '3.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:16,741::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9c3b180-a37d-4b41-96c7-ff1f0204c3ff`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416409', 'lastCheck': '3.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:16,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9c3b180-a37d-4b41-96c7-ff1f0204c3ff`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:16,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:16,741::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:16,742::task::993::Storage.TaskManager.Task::(_decref) Task=`b9c3b180-a37d-4b41-96c7-ff1f0204c3ff`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:17:16,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:16,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:16,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 15:17:16,919::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfbb2cd8-997a-40ae-a7b3-ff6cc272b90d`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:17:16,922::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:17:16,925::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:16,925::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:17:16,926::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:16,927::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:16,928::__init__::232::IOProcess::(_processLogs) (6694) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:17:16,930::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6694) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:16,932::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:16,932::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:16,932::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:16,932::__init__::232::IOProcess::(_processLogs) (6695) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:17:16,933::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6695) Queuing response >Thread-55::INFO::2014-08-11 15:17:16,933::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:17:16,934::task::1191::Storage.TaskManager.Task::(prepare) Task=`cfbb2cd8-997a-40ae-a7b3-ff6cc272b90d`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:17:16,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`cfbb2cd8-997a-40ae-a7b3-ff6cc272b90d`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:17:16,935::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:17:16,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:17:16,935::task::993::Storage.TaskManager.Task::(_decref) Task=`cfbb2cd8-997a-40ae-a7b3-ff6cc272b90d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:17:17,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:17,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131573 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:19,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:19,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136426 s, 75.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:21,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:21,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132868 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:17:21,886::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:17:21,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`945919e0-32c8-4294-921d-4be3de6a9541`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:21,920::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:17:21,934::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:21,934::task::1191::Storage.TaskManager.Task::(prepare) Task=`945919e0-32c8-4294-921d-4be3de6a9541`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:21,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`945919e0-32c8-4294-921d-4be3de6a9541`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:21,935::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:21,935::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:21,935::task::993::Storage.TaskManager.Task::(_decref) Task=`945919e0-32c8-4294-921d-4be3de6a9541`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:21,937::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:21,938::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:21,939::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:17:21,945::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:17:21,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc557fcb-3ada-4d6a-bca9-40eeaddcb637`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:17:21,946::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:17:21,947::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7ef683df-c9f8-4d10-88e2-8746853df400`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:17:21,947::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:17:21,948::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:17:21,949::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7ef683df-c9f8-4d10-88e2-8746853df400`::Granted request >Thread-34::DEBUG::2014-08-11 15:17:21,950::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bc557fcb-3ada-4d6a-bca9-40eeaddcb637`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:17:21,950::task::993::Storage.TaskManager.Task::(_decref) Task=`bc557fcb-3ada-4d6a-bca9-40eeaddcb637`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:17:21,970::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855242764288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:21,970::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc557fcb-3ada-4d6a-bca9-40eeaddcb637`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855242764288', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:21,970::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc557fcb-3ada-4d6a-bca9-40eeaddcb637`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:17:21,970::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:17:21,971::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:17:21,971::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:17:21,971::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:17:21,971::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:17:21,972::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:17:21,972::task::993::Storage.TaskManager.Task::(_decref) Task=`bc557fcb-3ada-4d6a-bca9-40eeaddcb637`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:17:23,228::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:23,229::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:23,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:23,229::__init__::232::IOProcess::(_processLogs) (6696) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:23,230::__init__::232::IOProcess::(_processLogs) (6696) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:23,230::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:17:23,249::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419949 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:23,250::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:23,250::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:23,251::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:23,251::__init__::232::IOProcess::(_processLogs) (6697) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:23,252::__init__::232::IOProcess::(_processLogs) (6697) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:23,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:23,253::__init__::232::IOProcess::(_processLogs) (6698) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:23,254::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:23,254::__init__::232::IOProcess::(_processLogs) (6699) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:23,255::__init__::232::IOProcess::(_processLogs) (6699) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:23,256::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:23,256::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:23,256::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:23,257::__init__::232::IOProcess::(_processLogs) (6700) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:23,257::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6700) Queuing response >Dummy-33::DEBUG::2014-08-11 15:17:23,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:23,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131993 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:25,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:25,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132368 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:26,946::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:26,948::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:27,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:17:27,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:27,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132227 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:29,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:29,492::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131746 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:31,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:31,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0195586 s, 52.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:17:31,852::task::595::Storage.TaskManager.Task::(_updateState) Task=`13704200-b451-426c-9f36-f527055650a9`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:31,853::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:17:31,853::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419949', 'lastCheck': '8.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:31,853::task::1191::Storage.TaskManager.Task::(prepare) Task=`13704200-b451-426c-9f36-f527055650a9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419949', 'lastCheck': '8.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:31,854::task::595::Storage.TaskManager.Task::(_updateState) Task=`13704200-b451-426c-9f36-f527055650a9`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:31,854::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:31,854::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:31,854::task::993::Storage.TaskManager.Task::(_decref) Task=`13704200-b451-426c-9f36-f527055650a9`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:17:31,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:17:31,985::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:17:31,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`37602234-869d-4a34-af1d-ea92e811bd0f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:32,021::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:32,027::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:32,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:32,030::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:17:32,037::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:32,039::task::1191::Storage.TaskManager.Task::(prepare) Task=`37602234-869d-4a34-af1d-ea92e811bd0f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:32,040::task::595::Storage.TaskManager.Task::(_updateState) Task=`37602234-869d-4a34-af1d-ea92e811bd0f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:32,041::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:32,043::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:32,044::task::993::Storage.TaskManager.Task::(_decref) Task=`37602234-869d-4a34-af1d-ea92e811bd0f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:17:32,058::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:17:32,060::task::595::Storage.TaskManager.Task::(_updateState) Task=`542eee46-5c77-4b66-bdcd-aa9c532a1b3a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:17:32,062::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:17:32,066::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`afaecd38-a704-417c-b5f4-815a9484aff6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:17:32,067::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:17:32,069::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:17:32,070::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`afaecd38-a704-417c-b5f4-815a9484aff6`::Granted request >Thread-34::DEBUG::2014-08-11 15:17:32,074::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`542eee46-5c77-4b66-bdcd-aa9c532a1b3a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:17:32,075::task::993::Storage.TaskManager.Task::(_decref) Task=`542eee46-5c77-4b66-bdcd-aa9c532a1b3a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:17:32,128::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855200296960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:32,129::task::1191::Storage.TaskManager.Task::(prepare) Task=`542eee46-5c77-4b66-bdcd-aa9c532a1b3a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855200296960', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:32,130::task::595::Storage.TaskManager.Task::(_updateState) Task=`542eee46-5c77-4b66-bdcd-aa9c532a1b3a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:17:32,131::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:17:32,132::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:17:32,134::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:17:32,136::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:17:32,137::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:17:32,138::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:17:32,140::task::993::Storage.TaskManager.Task::(_decref) Task=`542eee46-5c77-4b66-bdcd-aa9c532a1b3a`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:17:33,259::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:33,259::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:33,260::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:33,260::__init__::232::IOProcess::(_processLogs) (6701) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:33,260::__init__::232::IOProcess::(_processLogs) (6701) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:33,261::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:17:33,280::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424722 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:33,281::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:33,282::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:33,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:33,282::__init__::232::IOProcess::(_processLogs) (6702) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:33,283::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6702) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:33,284::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:33,285::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:33,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:33,286::__init__::232::IOProcess::(_processLogs) (6703) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:33,286::__init__::232::IOProcess::(_processLogs) (6703) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:33,286::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:33,287::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:33,287::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:33,287::__init__::232::IOProcess::(_processLogs) (6704) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:33,288::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6704) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:33,289::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:33,289::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:33,290::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:33,291::__init__::232::IOProcess::(_processLogs) (6705) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:33,291::__init__::232::IOProcess::(_processLogs) (6705) Queuing response >Dummy-33::DEBUG::2014-08-11 15:17:33,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:33,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132047 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:35,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:35,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260116 s, 39.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:37,033::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:37,035::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:37,121::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:17:37,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:37,901::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132065 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:39,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:39,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136181 s, 75.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:41,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:42,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131696 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:42,128::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:42,130::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:17:42,166::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:17:42,224::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bde3524-af60-4068-9d78-603d85abb71d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:42,226::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:42,226::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:17:42,240::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:42,240::task::1191::Storage.TaskManager.Task::(prepare) Task=`9bde3524-af60-4068-9d78-603d85abb71d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:42,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bde3524-af60-4068-9d78-603d85abb71d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:42,241::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:42,241::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:42,241::task::993::Storage.TaskManager.Task::(_decref) Task=`9bde3524-af60-4068-9d78-603d85abb71d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:17:42,252::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:17:42,252::task::595::Storage.TaskManager.Task::(_updateState) Task=`9464a30e-115d-471a-8a78-7f5f96fcad63`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:17:42,253::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:17:42,253::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8923b181-9edb-4881-b56d-8c4a268588b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:17:42,254::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:17:42,254::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:17:42,254::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8923b181-9edb-4881-b56d-8c4a268588b9`::Granted request >Thread-34::DEBUG::2014-08-11 15:17:42,255::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9464a30e-115d-471a-8a78-7f5f96fcad63`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:17:42,255::task::993::Storage.TaskManager.Task::(_decref) Task=`9464a30e-115d-471a-8a78-7f5f96fcad63`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:17:42,275::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855147868160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:42,275::task::1191::Storage.TaskManager.Task::(prepare) Task=`9464a30e-115d-471a-8a78-7f5f96fcad63`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855147868160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:42,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`9464a30e-115d-471a-8a78-7f5f96fcad63`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:17:42,276::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:17:42,276::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:17:42,276::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:17:42,277::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:17:42,277::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:17:42,277::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:17:42,277::task::993::Storage.TaskManager.Task::(_decref) Task=`9464a30e-115d-471a-8a78-7f5f96fcad63`::ref 0 aborting False >Thread-16::DEBUG::2014-08-11 15:17:43,290::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:17:43,292::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,293::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,293::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,293::__init__::232::IOProcess::(_processLogs) (6706) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:17:43,294::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6706) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:43,302::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:17:43,303::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,303::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,303::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,304::__init__::232::IOProcess::(_processLogs) (6707) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:17:43,305::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6707) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:43,305::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:17:43,306::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,306::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,306::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,307::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6708) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:17:43,307::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6708) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:43,308::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:17:43,309::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,309::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,309::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,309::__init__::232::IOProcess::(_processLogs) (6709) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:43,310::__init__::232::IOProcess::(_processLogs) (6709) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:43,310::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,311::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,311::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,311::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6710) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:17:43,311::__init__::232::IOProcess::(_processLogs) (6710) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:43,312::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:17:43,312::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:17:43,312::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:17:43,313::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,313::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,314::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,314::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|(6711) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:43,314::__init__::232::IOProcess::(_processLogs) (6711) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:43,315::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:17:43,333::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00048242 s, 943 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:43,335::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,335::__init__::232::IOProcess::(_processLogs) (6699) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,335::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,336::__init__::232::IOProcess::(_processLogs) (6712) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:43,336::__init__::232::IOProcess::(_processLogs) DEBUG|(6712) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:43,337::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,337::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,338::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,338::__init__::232::IOProcess::(_processLogs) DEBUG|(6713) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:43,338::__init__::232::IOProcess::(_processLogs) (6713) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:43,339::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,339::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,340::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,341::__init__::232::IOProcess::(_processLogs) (6714) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:43,341::__init__::232::IOProcess::(_processLogs) (6714) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:43,341::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:43,342::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:43,343::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:43,343::__init__::232::IOProcess::(_processLogs) (6715) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:43,343::__init__::232::IOProcess::(_processLogs) (6715) Queuing response >Dummy-33::DEBUG::2014-08-11 15:17:44,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:44,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01338 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:46,090::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:46,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132724 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:17:47,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`b14967cc-d196-430e-bdb7-02c5acdb44c6`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:47,023::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:17:47,024::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00048242', 'lastCheck': '3.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:47,024::task::1191::Storage.TaskManager.Task::(prepare) Task=`b14967cc-d196-430e-bdb7-02c5acdb44c6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00048242', 'lastCheck': '3.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:17:47,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`b14967cc-d196-430e-bdb7-02c5acdb44c6`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:47,025::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:47,025::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:47,025::task::993::Storage.TaskManager.Task::(_decref) Task=`b14967cc-d196-430e-bdb7-02c5acdb44c6`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:17:47,046::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:47,230::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:47,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:47,232::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:17:48,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:48,179::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133269 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:50,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:50,232::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132727 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:52,235::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:52,237::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:17:52,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-115::DEBUG::2014-08-11 15:17:52,291::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:17:52,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`63704f88-df69-4808-8917-3f415e719303`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:17:52,345::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:52,346::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:17:52,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0183326 s, 55.9 MB/s\n'; <rc> = 0 >Thread-115::INFO::2014-08-11 15:17:52,368::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:52,369::task::1191::Storage.TaskManager.Task::(prepare) Task=`63704f88-df69-4808-8917-3f415e719303`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:17:52,370::task::595::Storage.TaskManager.Task::(_updateState) Task=`63704f88-df69-4808-8917-3f415e719303`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:17:52,371::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:17:52,372::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:17:52,373::task::993::Storage.TaskManager.Task::(_decref) Task=`63704f88-df69-4808-8917-3f415e719303`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:17:52,387::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:17:52,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`52f81c6f-4406-4d15-b88e-76e096dea545`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:17:52,390::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:17:52,393::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7544c00b-2250-42af-b83d-9a66c6f1d987`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:17:52,394::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:17:52,395::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:17:52,396::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7544c00b-2250-42af-b83d-9a66c6f1d987`::Granted request >Thread-34::DEBUG::2014-08-11 15:17:52,399::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`52f81c6f-4406-4d15-b88e-76e096dea545`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:17:52,400::task::993::Storage.TaskManager.Task::(_decref) Task=`52f81c6f-4406-4d15-b88e-76e096dea545`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:17:52,455::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855104155648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:52,455::task::1191::Storage.TaskManager.Task::(prepare) Task=`52f81c6f-4406-4d15-b88e-76e096dea545`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855104155648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:17:52,456::task::595::Storage.TaskManager.Task::(_updateState) Task=`52f81c6f-4406-4d15-b88e-76e096dea545`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:17:52,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:17:52,456::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:17:52,456::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:17:52,457::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:17:52,457::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:17:52,457::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:17:52,457::task::993::Storage.TaskManager.Task::(_decref) Task=`52f81c6f-4406-4d15-b88e-76e096dea545`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:17:53,344::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:53,345::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:17:53,346::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:17:53,346::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:53,365::__init__::232::IOProcess::(_processLogs) (6716) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:53,367::__init__::232::IOProcess::(_processLogs) (6716) Queuing response >Thread-16::DEBUG::2014-08-11 15:17:53,368::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000568357 s, 801 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:17:53,370::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:53,370::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:53,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:53,370::__init__::232::IOProcess::(_processLogs) (6717) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:17:53,371::__init__::232::IOProcess::(_processLogs) DEBUG|(6717) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:53,373::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:53,373::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:53,374::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:53,374::__init__::232::IOProcess::(_processLogs) (6718) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:53,374::__init__::232::IOProcess::(_processLogs) (6718) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:53,375::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:53,375::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:53,376::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:53,376::__init__::232::IOProcess::(_processLogs) (6719) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:53,376::__init__::232::IOProcess::(_processLogs) (6719) Queuing response >Thread-14::DEBUG::2014-08-11 15:17:53,377::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:17:53,377::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:17:53,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:17:53,379::__init__::232::IOProcess::(_processLogs) (6720) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:17:53,379::__init__::232::IOProcess::(_processLogs) (6720) Queuing response >Dummy-33::DEBUG::2014-08-11 15:17:54,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:54,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131505 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:17:56,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:56,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135722 s, 75.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:57,352::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:57,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:17:57,422::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:17:58,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:17:58,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132948 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:00,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:00,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013467 s, 76.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:18:02,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`37ad6700-37a8-4c10-b90d-ce0c6391e172`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:02,138::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:18:02,140::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000568357', 'lastCheck': '8.8', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:02,142::task::1191::Storage.TaskManager.Task::(prepare) Task=`37ad6700-37a8-4c10-b90d-ce0c6391e172`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000568357', 'lastCheck': '8.8', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:02,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`37ad6700-37a8-4c10-b90d-ce0c6391e172`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:02,144::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:02,146::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:02,147::task::993::Storage.TaskManager.Task::(_decref) Task=`37ad6700-37a8-4c10-b90d-ce0c6391e172`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:18:02,208::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:02,431::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:02,437::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:02,442::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:18:02,471::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:18:02,473::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f7e82ca-8b8f-4a8c-aa69-8900b9c3ea6a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:02,475::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:18:02,494::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:02,496::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f7e82ca-8b8f-4a8c-aa69-8900b9c3ea6a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:02,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f7e82ca-8b8f-4a8c-aa69-8900b9c3ea6a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:02,498::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:02,499::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:02,500::task::993::Storage.TaskManager.Task::(_decref) Task=`0f7e82ca-8b8f-4a8c-aa69-8900b9c3ea6a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:18:02,516::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:18:02,519::task::595::Storage.TaskManager.Task::(_updateState) Task=`c76cc55c-5de2-4037-a1ca-36eeefd3a6f6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:18:02,520::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:18:02,524::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30f5573b-3d78-4adb-8241-5977c9a32d5e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:18:02,525::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:18:02,528::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:18:02,529::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30f5573b-3d78-4adb-8241-5977c9a32d5e`::Granted request >Thread-34::DEBUG::2014-08-11 15:18:02,532::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c76cc55c-5de2-4037-a1ca-36eeefd3a6f6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:18:02,534::task::993::Storage.TaskManager.Task::(_decref) Task=`c76cc55c-5de2-4037-a1ca-36eeefd3a6f6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:18:02,589::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855056248832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:02,591::task::1191::Storage.TaskManager.Task::(prepare) Task=`c76cc55c-5de2-4037-a1ca-36eeefd3a6f6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855056248832', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:02,592::task::595::Storage.TaskManager.Task::(_updateState) Task=`c76cc55c-5de2-4037-a1ca-36eeefd3a6f6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:18:02,593::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:18:02,595::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:18:02,597::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:18:02,600::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:18:02,601::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:18:02,602::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:18:02,605::task::993::Storage.TaskManager.Task::(_decref) Task=`c76cc55c-5de2-4037-a1ca-36eeefd3a6f6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:18:02,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:02,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0294677 s, 34.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:03,379::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:03,380::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:18:03,381::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:18:03,381::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:03,400::__init__::232::IOProcess::(_processLogs) (6721) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:03,401::__init__::232::IOProcess::(_processLogs) (6721) Queuing response >Thread-16::DEBUG::2014-08-11 15:18:03,402::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000508133 s, 895 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:03,403::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:03,403::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:03,404::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:03,404::__init__::232::IOProcess::(_processLogs) (6722) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:03,405::__init__::232::IOProcess::(_processLogs) DEBUG|(6722) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:03,406::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:03,407::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:03,407::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:03,408::__init__::232::IOProcess::(_processLogs) (6723) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:03,408::__init__::232::IOProcess::(_processLogs) (6723) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:03,408::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:03,409::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:03,409::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:03,410::__init__::232::IOProcess::(_processLogs) (6724) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:03,410::__init__::232::IOProcess::(_processLogs) (6724) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:03,410::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:03,411::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:03,412::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:03,412::__init__::232::IOProcess::(_processLogs) (6725) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:03,413::__init__::232::IOProcess::(_processLogs) (6725) Queuing response >Dummy-33::DEBUG::2014-08-11 15:18:04,924::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:04,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013212 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:06,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:07,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137038 s, 74.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:07,447::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:07,449::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:07,522::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:18:09,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:09,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132213 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:11,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:11,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131799 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:12,528::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:12,529::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:12,530::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:18:12,621::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:18:12,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`085e20b1-5896-478f-8017-ede8c5460130`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:12,622::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:18:12,635::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:12,635::task::1191::Storage.TaskManager.Task::(prepare) Task=`085e20b1-5896-478f-8017-ede8c5460130`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:12,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`085e20b1-5896-478f-8017-ede8c5460130`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:12,636::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:12,636::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:12,636::task::993::Storage.TaskManager.Task::(_decref) Task=`085e20b1-5896-478f-8017-ede8c5460130`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:18:12,646::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:18:12,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b7da128-f564-4d4f-8b14-5697b7f30883`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:18:12,647::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:18:12,647::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a7318989-65d7-4f01-a026-68ef3324936d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:18:12,648::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:18:12,648::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:18:12,648::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a7318989-65d7-4f01-a026-68ef3324936d`::Granted request >Thread-34::DEBUG::2014-08-11 15:18:12,649::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9b7da128-f564-4d4f-8b14-5697b7f30883`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:18:12,649::task::993::Storage.TaskManager.Task::(_decref) Task=`9b7da128-f564-4d4f-8b14-5697b7f30883`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:18:12,669::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855013584896', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:12,669::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b7da128-f564-4d4f-8b14-5697b7f30883`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1855013584896', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:12,669::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b7da128-f564-4d4f-8b14-5697b7f30883`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:18:12,670::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:18:12,670::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:18:12,670::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:18:12,670::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:18:12,671::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:18:12,671::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:18:12,671::task::993::Storage.TaskManager.Task::(_decref) Task=`9b7da128-f564-4d4f-8b14-5697b7f30883`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:18:13,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:13,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140118 s, 73.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:13,413::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:13,414::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:18:13,415::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:18:13,415::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:13,434::__init__::232::IOProcess::(_processLogs) (6726) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:13,435::__init__::232::IOProcess::(_processLogs) (6726) Queuing response >Thread-16::DEBUG::2014-08-11 15:18:13,437::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000503472 s, 904 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:13,438::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:13,438::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:13,438::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:13,439::__init__::232::IOProcess::(_processLogs) (6727) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:13,439::__init__::232::IOProcess::(_processLogs) DEBUG|(6727) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:13,441::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:13,441::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:13,442::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:13,442::__init__::232::IOProcess::(_processLogs) (6728) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:13,442::__init__::232::IOProcess::(_processLogs) (6728) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:13,443::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:13,443::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:13,444::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:13,444::__init__::232::IOProcess::(_processLogs) (6729) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:13,444::__init__::232::IOProcess::(_processLogs) (6729) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:13,445::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:13,446::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:13,447::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:13,447::__init__::232::IOProcess::(_processLogs) (6730) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:13,447::__init__::232::IOProcess::(_processLogs) (6730) Queuing response >Dummy-33::DEBUG::2014-08-11 15:18:15,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:15,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132141 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:17,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:17,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133378 s, 76.8 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:18:17,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`94e19412-a2ce-424c-80e8-8f07d7d8ef4b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:17,319::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:18:17,321::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000503472', 'lastCheck': '3.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:17,322::task::1191::Storage.TaskManager.Task::(prepare) Task=`94e19412-a2ce-424c-80e8-8f07d7d8ef4b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000503472', 'lastCheck': '3.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:17,323::task::595::Storage.TaskManager.Task::(_updateState) Task=`94e19412-a2ce-424c-80e8-8f07d7d8ef4b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:17,324::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:17,325::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:17,326::task::993::Storage.TaskManager.Task::(_decref) Task=`94e19412-a2ce-424c-80e8-8f07d7d8ef4b`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:18:17,358::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-55::DEBUG::2014-08-11 15:18:17,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`891e9c8a-9368-4d8c-af38-edf4bc49dec8`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:18:17,438::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:18:17,439::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:17,439::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:17,439::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:17,440::__init__::232::IOProcess::(_processLogs) (6731) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:18:17,440::__init__::232::IOProcess::(_processLogs) DEBUG|(6731) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:17,441::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:17,441::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:17,442::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:17,442::__init__::232::IOProcess::(_processLogs) DEBUG|(6732) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:18:17,442::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:18:17,443::__init__::232::IOProcess::(_processLogs) (6732) Queuing response >Thread-55::DEBUG::2014-08-11 15:18:17,444::task::1191::Storage.TaskManager.Task::(prepare) Task=`891e9c8a-9368-4d8c-af38-edf4bc49dec8`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:18:17,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`891e9c8a-9368-4d8c-af38-edf4bc49dec8`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:18:17,445::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:18:17,445::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:18:17,446::task::993::Storage.TaskManager.Task::(_decref) Task=`891e9c8a-9368-4d8c-af38-edf4bc49dec8`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:17,537::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:17,538::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:17,623::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:18:19,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:19,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132756 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:21,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:21,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133409 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:22,629::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:22,630::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:22,631::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:18:22,684::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:18:22,685::task::595::Storage.TaskManager.Task::(_updateState) Task=`66da87cb-08f6-4383-993a-55b34ec51058`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:22,685::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:18:22,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:22,699::task::1191::Storage.TaskManager.Task::(prepare) Task=`66da87cb-08f6-4383-993a-55b34ec51058`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:22,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`66da87cb-08f6-4383-993a-55b34ec51058`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:22,700::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:22,700::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:22,700::task::993::Storage.TaskManager.Task::(_decref) Task=`66da87cb-08f6-4383-993a-55b34ec51058`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:18:22,713::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:18:22,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`4872828e-5814-4458-bdf8-ae07d6b13f07`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:18:22,714::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:18:22,714::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`72a9d4f1-f18c-4162-8374-d99c7647f61f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:18:22,715::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:18:22,715::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:18:22,715::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`72a9d4f1-f18c-4162-8374-d99c7647f61f`::Granted request >Thread-34::DEBUG::2014-08-11 15:18:22,716::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4872828e-5814-4458-bdf8-ae07d6b13f07`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:18:22,716::task::993::Storage.TaskManager.Task::(_decref) Task=`4872828e-5814-4458-bdf8-ae07d6b13f07`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:18:22,736::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854961025024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:22,736::task::1191::Storage.TaskManager.Task::(prepare) Task=`4872828e-5814-4458-bdf8-ae07d6b13f07`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854961025024', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:22,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`4872828e-5814-4458-bdf8-ae07d6b13f07`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:18:22,737::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:18:22,737::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:18:22,738::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:18:22,738::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:18:22,738::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:18:22,738::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:18:22,739::task::993::Storage.TaskManager.Task::(_decref) Task=`4872828e-5814-4458-bdf8-ae07d6b13f07`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:18:23,447::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:23,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:23,449::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:18:23,450::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:18:23,450::__init__::232::IOProcess::(_processLogs) (6733) Got request for method 'statvfs' >Dummy-33::DEBUG::2014-08-11 15:18:23,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:18:23,488::__init__::232::IOProcess::(_processLogs) (6733) Queuing response >Thread-16::DEBUG::2014-08-11 15:18:23,489::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00750963 s, 60.6 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:23,491::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:23,491::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:23,492::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:23,492::__init__::232::IOProcess::(_processLogs) (6734) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:23,501::__init__::232::IOProcess::(_processLogs) DEBUG|(6734) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:23,504::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:23,504::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:23,504::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:23,506::__init__::232::IOProcess::(_processLogs) (6735) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:23,508::__init__::232::IOProcess::(_processLogs) DEBUG|(6735) Queuing response >Dummy-33::DEBUG::2014-08-11 15:18:23,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0241445 s, 42.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:23,518::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:23,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:23,520::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:23,521::__init__::232::IOProcess::(_processLogs) (6736) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:23,523::__init__::232::IOProcess::(_processLogs) (6736) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:23,525::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:23,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:23,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:23,528::__init__::232::IOProcess::(_processLogs) (6737) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:23,530::__init__::232::IOProcess::(_processLogs) DEBUG|(6737) Queuing response >Dummy-33::DEBUG::2014-08-11 15:18:25,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:25,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133101 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:27,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:27,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:27,645::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:27,651::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:18:27,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027077 s, 37.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:29,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:29,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134239 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:31,735::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:31,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133676 s, 76.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:18:32,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`350fa207-dad2-45d1-a0d5-3f2678d77de2`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:32,449::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:18:32,451::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00750963', 'lastCheck': '8.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:32,453::task::1191::Storage.TaskManager.Task::(prepare) Task=`350fa207-dad2-45d1-a0d5-3f2678d77de2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00750963', 'lastCheck': '8.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:32,454::task::595::Storage.TaskManager.Task::(_updateState) Task=`350fa207-dad2-45d1-a0d5-3f2678d77de2`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:32,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:32,457::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:32,459::task::993::Storage.TaskManager.Task::(_decref) Task=`350fa207-dad2-45d1-a0d5-3f2678d77de2`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:18:32,507::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:32,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:32,669::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:32,727::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:18:32,754::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:18:32,757::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6c812aa-2a33-4a6a-ae77-d549db9c96be`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:32,759::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:18:32,773::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:32,774::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6c812aa-2a33-4a6a-ae77-d549db9c96be`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:32,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6c812aa-2a33-4a6a-ae77-d549db9c96be`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:32,777::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:32,778::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:32,780::task::993::Storage.TaskManager.Task::(_decref) Task=`e6c812aa-2a33-4a6a-ae77-d549db9c96be`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:18:32,796::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:18:32,799::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7941318-5413-4642-b8a9-273cb62ec60d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:18:32,800::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:18:32,803::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cdbbf3d3-6843-4f98-971a-801228832d45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:18:32,805::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:18:32,807::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:18:32,808::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cdbbf3d3-6843-4f98-971a-801228832d45`::Granted request >Thread-34::DEBUG::2014-08-11 15:18:32,811::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d7941318-5413-4642-b8a9-273cb62ec60d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:18:32,813::task::993::Storage.TaskManager.Task::(_decref) Task=`d7941318-5413-4642-b8a9-273cb62ec60d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:18:32,884::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854916001792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:32,886::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7941318-5413-4642-b8a9-273cb62ec60d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854916001792', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:32,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7941318-5413-4642-b8a9-273cb62ec60d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:18:32,888::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:18:32,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:18:32,892::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:18:32,894::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:18:32,896::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:18:32,897::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:18:32,899::task::993::Storage.TaskManager.Task::(_decref) Task=`d7941318-5413-4642-b8a9-273cb62ec60d`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:18:33,531::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:33,532::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:33,532::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:33,532::__init__::232::IOProcess::(_processLogs) (6738) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:33,533::__init__::232::IOProcess::(_processLogs) DEBUG|(6738) Queuing response >Thread-16::DEBUG::2014-08-11 15:18:33,534::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:18:33,553::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432343 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:33,554::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:33,554::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:33,555::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:33,555::__init__::232::IOProcess::(_processLogs) (6739) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:33,556::__init__::232::IOProcess::(_processLogs) DEBUG|(6739) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:33,557::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:33,558::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:33,558::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:33,558::__init__::232::IOProcess::(_processLogs) (6740) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:33,558::__init__::232::IOProcess::(_processLogs) (6740) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:33,559::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:33,559::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:33,559::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:33,559::__init__::232::IOProcess::(_processLogs) (6741) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:33,560::__init__::232::IOProcess::(_processLogs) DEBUG|(6741) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:33,561::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:33,562::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:33,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:33,563::__init__::232::IOProcess::(_processLogs) (6742) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:33,563::__init__::232::IOProcess::(_processLogs) (6742) Queuing response >Dummy-33::DEBUG::2014-08-11 15:18:33,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:33,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132439 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:35,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:35,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132767 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:37,738::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:37,739::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:37,823::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:18:37,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:37,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132839 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:39,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:39,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013241 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:42,015::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:42,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133021 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:42,829::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:42,831::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:42,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:18:42,915::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:18:42,916::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e6ea032-9ac4-4544-b481-c9fa2d485a81`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:42,916::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:18:42,929::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:42,930::task::1191::Storage.TaskManager.Task::(prepare) Task=`3e6ea032-9ac4-4544-b481-c9fa2d485a81`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:42,930::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e6ea032-9ac4-4544-b481-c9fa2d485a81`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:42,930::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:42,930::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:42,930::task::993::Storage.TaskManager.Task::(_decref) Task=`3e6ea032-9ac4-4544-b481-c9fa2d485a81`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:18:42,943::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:18:42,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`25f27ff4-8390-4eff-9087-25a8fc474d3a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:18:42,944::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:18:42,945::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4ad16ed7-29d1-4f26-bdca-3fac50826555`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:18:42,945::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:18:42,945::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:18:42,946::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4ad16ed7-29d1-4f26-bdca-3fac50826555`::Granted request >Thread-34::DEBUG::2014-08-11 15:18:42,946::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`25f27ff4-8390-4eff-9087-25a8fc474d3a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:18:42,946::task::993::Storage.TaskManager.Task::(_decref) Task=`25f27ff4-8390-4eff-9087-25a8fc474d3a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:18:42,966::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854838931456', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:42,966::task::1191::Storage.TaskManager.Task::(prepare) Task=`25f27ff4-8390-4eff-9087-25a8fc474d3a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854838931456', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:42,967::task::595::Storage.TaskManager.Task::(_updateState) Task=`25f27ff4-8390-4eff-9087-25a8fc474d3a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:18:42,967::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:18:42,967::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:18:42,967::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:18:42,968::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:18:42,968::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:18:42,968::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:18:42,969::task::993::Storage.TaskManager.Task::(_decref) Task=`25f27ff4-8390-4eff-9087-25a8fc474d3a`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:18:43,563::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:43,564::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:43,564::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:43,564::__init__::232::IOProcess::(_processLogs) (6743) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:43,565::__init__::232::IOProcess::(_processLogs) (6743) Queuing response >Thread-16::DEBUG::2014-08-11 15:18:43,565::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:18:43,584::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000677031 s, 672 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:43,585::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:43,586::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:43,586::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:43,586::__init__::232::IOProcess::(_processLogs) (6744) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:43,587::__init__::232::IOProcess::(_processLogs) (6744) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:43,588::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:43,589::__init__::232::IOProcess::(_processLogs) (6745) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:43,590::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:43,590::__init__::232::IOProcess::(_processLogs) (6746) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:43,590::__init__::232::IOProcess::(_processLogs) (6746) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:43,591::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:43,592::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:43,592::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:43,593::__init__::232::IOProcess::(_processLogs) (6747) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:43,593::__init__::232::IOProcess::(_processLogs) (6747) Queuing response >Dummy-33::DEBUG::2014-08-11 15:18:44,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:44,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130856 s, 78.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:46,127::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:46,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133847 s, 76.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:18:47,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecadfd19-105b-42b6-be33-fc39dffa465b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:47,612::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:18:47,613::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000677031', 'lastCheck': '4.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:47,613::task::1191::Storage.TaskManager.Task::(prepare) Task=`ecadfd19-105b-42b6-be33-fc39dffa465b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000677031', 'lastCheck': '4.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:18:47,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`ecadfd19-105b-42b6-be33-fc39dffa465b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:47,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:47,614::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:47,614::task::993::Storage.TaskManager.Task::(_decref) Task=`ecadfd19-105b-42b6-be33-fc39dffa465b`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:18:47,635::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:47,835::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:47,836::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:47,923::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:18:48,203::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:48,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013561 s, 75.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:50,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:50,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133511 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:52,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:52,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132195 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:52,930::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:52,932::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:18:52,987::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:18:53,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4e96b7d-bc43-4b79-bb9c-5b95c082348f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:18:53,024::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:53,026::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:18:53,037::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:53,037::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4e96b7d-bc43-4b79-bb9c-5b95c082348f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:18:53,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4e96b7d-bc43-4b79-bb9c-5b95c082348f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:18:53,038::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:18:53,038::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:18:53,038::task::993::Storage.TaskManager.Task::(_decref) Task=`f4e96b7d-bc43-4b79-bb9c-5b95c082348f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:18:53,050::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:18:53,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e9b0d1a-d708-4866-adba-0dbc5cb6cf5e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:18:53,051::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:18:53,052::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4f3ae171-6843-4a9f-b587-252190327076`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:18:53,052::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:18:53,052::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:18:53,053::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4f3ae171-6843-4a9f-b587-252190327076`::Granted request >Thread-34::DEBUG::2014-08-11 15:18:53,053::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3e9b0d1a-d708-4866-adba-0dbc5cb6cf5e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:18:53,053::task::993::Storage.TaskManager.Task::(_decref) Task=`3e9b0d1a-d708-4866-adba-0dbc5cb6cf5e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:18:53,074::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854792794112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:53,076::task::1191::Storage.TaskManager.Task::(prepare) Task=`3e9b0d1a-d708-4866-adba-0dbc5cb6cf5e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854792794112', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:18:53,077::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e9b0d1a-d708-4866-adba-0dbc5cb6cf5e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:18:53,078::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:18:53,080::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:18:53,082::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:18:53,084::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:18:53,085::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:18:53,086::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:18:53,089::task::993::Storage.TaskManager.Task::(_decref) Task=`3e9b0d1a-d708-4866-adba-0dbc5cb6cf5e`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:18:53,594::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:53,594::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:53,595::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:53,595::__init__::232::IOProcess::(_processLogs) (6748) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:53,595::__init__::232::IOProcess::(_processLogs) (6748) Queuing response >Thread-16::DEBUG::2014-08-11 15:18:53,596::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:18:53,615::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418618 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:18:53,616::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:53,616::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:53,617::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:53,617::__init__::232::IOProcess::(_processLogs) (6749) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:18:53,617::__init__::232::IOProcess::(_processLogs) (6749) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:53,618::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:53,619::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:53,620::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:53,620::__init__::232::IOProcess::(_processLogs) (6750) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:53,620::__init__::232::IOProcess::(_processLogs) (6750) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:53,620::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:53,621::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:53,622::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:53,622::__init__::232::IOProcess::(_processLogs) (6751) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:53,622::__init__::232::IOProcess::(_processLogs) (6751) Queuing response >Thread-14::DEBUG::2014-08-11 15:18:53,623::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:18:53,623::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:18:53,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:18:53,625::__init__::232::IOProcess::(_processLogs) (6752) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:18:53,625::__init__::232::IOProcess::(_processLogs) (6752) Queuing response >Dummy-33::DEBUG::2014-08-11 15:18:54,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:54,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131577 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:18:56,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:56,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133007 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:58,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:58,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:18:58,123::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:18:58,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:18:58,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132696 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:00,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:00,588::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134252 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:02,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:02,661::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268371 s, 38.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:19:02,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`45a46860-6ff8-4ec8-9eb9-af93dc3aa811`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:02,736::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:19:02,738::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418618', 'lastCheck': '9.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:02,739::task::1191::Storage.TaskManager.Task::(prepare) Task=`45a46860-6ff8-4ec8-9eb9-af93dc3aa811`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418618', 'lastCheck': '9.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:02,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`45a46860-6ff8-4ec8-9eb9-af93dc3aa811`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:02,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:02,743::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:02,744::task::993::Storage.TaskManager.Task::(_decref) Task=`45a46860-6ff8-4ec8-9eb9-af93dc3aa811`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:19:02,792::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:19:03,104::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:19:03,105::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f9c2cc0-f842-4b05-aa22-3feb3872adee`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:03,122::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:03,130::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:03,132::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:03,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:19:03,135::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:03,135::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f9c2cc0-f842-4b05-aa22-3feb3872adee`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:03,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f9c2cc0-f842-4b05-aa22-3feb3872adee`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:03,136::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:03,136::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:03,136::task::993::Storage.TaskManager.Task::(_decref) Task=`2f9c2cc0-f842-4b05-aa22-3feb3872adee`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:19:03,148::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:19:03,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`e64ce495-3b90-4377-8619-a2a7fe8b8ae1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:19:03,149::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:19:03,150::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`69735c28-3b9e-422f-a433-0317430bbe58`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:19:03,150::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:19:03,150::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:19:03,150::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`69735c28-3b9e-422f-a433-0317430bbe58`::Granted request >Thread-34::DEBUG::2014-08-11 15:19:03,151::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e64ce495-3b90-4377-8619-a2a7fe8b8ae1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:19:03,151::task::993::Storage.TaskManager.Task::(_decref) Task=`e64ce495-3b90-4377-8619-a2a7fe8b8ae1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:19:03,172::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854746591232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:03,173::task::1191::Storage.TaskManager.Task::(prepare) Task=`e64ce495-3b90-4377-8619-a2a7fe8b8ae1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854746591232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:03,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`e64ce495-3b90-4377-8619-a2a7fe8b8ae1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:19:03,175::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:19:03,176::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:19:03,178::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:19:03,180::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:19:03,181::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:19:03,182::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:19:03,185::task::993::Storage.TaskManager.Task::(_decref) Task=`e64ce495-3b90-4377-8619-a2a7fe8b8ae1`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:19:03,625::__init__::232::IOProcess::(_processLogs) (6746) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:19:03,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:19:03,626::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:03,626::__init__::232::IOProcess::(_processLogs) (6753) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:19:03,626::__init__::232::IOProcess::(_processLogs) (6753) Queuing response >Thread-16::DEBUG::2014-08-11 15:19:03,627::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:19:03,646::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000434982 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:19:03,647::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:03,647::__init__::232::IOProcess::(_processLogs) (6754) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:19:03,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:03,649::__init__::232::IOProcess::(_processLogs) (6755) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:03,650::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:03,650::__init__::232::IOProcess::(_processLogs) (6756) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:03,651::__init__::232::IOProcess::(_processLogs) (6756) Queuing response >Thread-14::DEBUG::2014-08-11 15:19:03,652::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:03,652::__init__::232::IOProcess::(_processLogs) (6757) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:19:04,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:04,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131342 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:06,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:07,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131636 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:08,138::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:08,139::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:08,223::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:19:09,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:09,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133204 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:11,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:11,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135004 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:13,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:13,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133147 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:19:13,204::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:19:13,225::task::595::Storage.TaskManager.Task::(_updateState) Task=`047aee28-377d-4bcb-a636-7bafdedc9906`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:13,226::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:13,232::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:13,242::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:19:13,244::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:13,246::task::1191::Storage.TaskManager.Task::(prepare) Task=`047aee28-377d-4bcb-a636-7bafdedc9906`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:13,248::task::595::Storage.TaskManager.Task::(_updateState) Task=`047aee28-377d-4bcb-a636-7bafdedc9906`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:13,249::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:19:13,249::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:13,251::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:13,252::task::993::Storage.TaskManager.Task::(_decref) Task=`047aee28-377d-4bcb-a636-7bafdedc9906`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:19:13,266::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:19:13,270::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e69921c-a64d-4724-8ee4-88f54bb40832`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:19:13,272::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:19:13,276::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1dd131cf-74ae-4c1a-8657-6f54315e25a6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:19:13,277::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:19:13,279::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:19:13,280::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1dd131cf-74ae-4c1a-8657-6f54315e25a6`::Granted request >Thread-34::DEBUG::2014-08-11 15:19:13,283::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4e69921c-a64d-4724-8ee4-88f54bb40832`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:19:13,285::task::993::Storage.TaskManager.Task::(_decref) Task=`4e69921c-a64d-4724-8ee4-88f54bb40832`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:19:13,347::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854687870976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:13,349::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e69921c-a64d-4724-8ee4-88f54bb40832`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854687870976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:13,350::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e69921c-a64d-4724-8ee4-88f54bb40832`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:19:13,352::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:19:13,353::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:19:13,355::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:19:13,357::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:19:13,358::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:19:13,359::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:19:13,362::task::993::Storage.TaskManager.Task::(_decref) Task=`4e69921c-a64d-4724-8ee4-88f54bb40832`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:19:13,654::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:13,654::__init__::232::IOProcess::(_processLogs) (6758) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:19:13,655::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:19:13,674::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000536319 s, 848 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:19:13,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:13,676::__init__::232::IOProcess::(_processLogs) (6759) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:19:13,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:13,677::__init__::232::IOProcess::(_processLogs) (6760) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:13,678::__init__::232::IOProcess::(_processLogs) (6760) Queuing response >Thread-14::DEBUG::2014-08-11 15:19:13,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:13,679::__init__::232::IOProcess::(_processLogs) (6761) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:13,679::__init__::232::IOProcess::(_processLogs) DEBUG|(6761) Queuing response >Thread-14::DEBUG::2014-08-11 15:19:13,680::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:19:13,680::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:19:13,680::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:13,681::__init__::232::IOProcess::(_processLogs) (6762) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:13,681::__init__::232::IOProcess::(_processLogs) (6762) Queuing response >Dummy-33::DEBUG::2014-08-11 15:19:15,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:15,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132674 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:17,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:17,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132582 s, 77.2 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:19:17,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ae4fb0a-7430-4911-90ca-11fa8d392749`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:19:17,598::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:19:17,599::__init__::232::IOProcess::(_processLogs) (6763) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:19:17,601::__init__::232::IOProcess::(_processLogs) (6764) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:19:17,601::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:19:17,601::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ae4fb0a-7430-4911-90ca-11fa8d392749`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:19:17,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ae4fb0a-7430-4911-90ca-11fa8d392749`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:19:17,602::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:19:17,602::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:19:17,602::task::993::Storage.TaskManager.Task::(_decref) Task=`7ae4fb0a-7430-4911-90ca-11fa8d392749`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:19:17,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3dace1d-ebd2-460b-8e1b-e6fe89c7ab54`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:17,892::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:19:17,892::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000536319', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:17,893::task::1191::Storage.TaskManager.Task::(prepare) Task=`b3dace1d-ebd2-460b-8e1b-e6fe89c7ab54`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000536319', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:17,893::task::595::Storage.TaskManager.Task::(_updateState) Task=`b3dace1d-ebd2-460b-8e1b-e6fe89c7ab54`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:17,893::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:17,893::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:17,894::task::993::Storage.TaskManager.Task::(_decref) Task=`b3dace1d-ebd2-460b-8e1b-e6fe89c7ab54`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:19:17,913::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:18,259::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:18,260::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:18,323::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:19:19,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:19,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134007 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:21,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:21,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132747 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:23,330::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:23,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:19:23,379::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:19:23,425::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fac3779-eafe-485b-9f2a-495537878627`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:23,426::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:23,427::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:19:23,441::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:23,443::task::1191::Storage.TaskManager.Task::(prepare) Task=`0fac3779-eafe-485b-9f2a-495537878627`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:23,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`0fac3779-eafe-485b-9f2a-495537878627`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:23,446::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:23,447::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:23,448::task::993::Storage.TaskManager.Task::(_decref) Task=`0fac3779-eafe-485b-9f2a-495537878627`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:19:23,464::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:19:23,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e6e07f9-4efa-4a04-90f5-b8f5467cb6d1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:19:23,469::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:19:23,473::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`89fdc059-2ab2-43c0-8761-bf408477e822`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:19:23,474::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:19:23,477::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:19:23,479::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`89fdc059-2ab2-43c0-8761-bf408477e822`::Granted request >Thread-34::DEBUG::2014-08-11 15:19:23,483::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7e6e07f9-4efa-4a04-90f5-b8f5467cb6d1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:19:23,485::task::993::Storage.TaskManager.Task::(_decref) Task=`7e6e07f9-4efa-4a04-90f5-b8f5467cb6d1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:19:23,543::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854650122240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:23,544::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e6e07f9-4efa-4a04-90f5-b8f5467cb6d1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854650122240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:23,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e6e07f9-4efa-4a04-90f5-b8f5467cb6d1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:19:23,547::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:19:23,548::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:19:23,550::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:19:23,552::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:19:23,553::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:19:23,554::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:19:23,557::task::993::Storage.TaskManager.Task::(_decref) Task=`7e6e07f9-4efa-4a04-90f5-b8f5467cb6d1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:19:23,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:23,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013074 s, 78.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:19:23,683::__init__::232::IOProcess::(_processLogs) (6765) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:19:23,684::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:19:23,703::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00043102 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:19:23,705::__init__::232::IOProcess::(_processLogs) (6766) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:19:23,706::__init__::232::IOProcess::(_processLogs) (6767) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:23,707::__init__::232::IOProcess::(_processLogs) (6768) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:23,708::__init__::232::IOProcess::(_processLogs) (6769) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:19:25,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:25,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131532 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:27,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:27,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132157 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:28,435::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:28,436::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:28,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:19:29,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:29,796::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133292 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:31,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:31,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133952 s, 76.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:19:33,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`87eb570e-92cb-4033-b94f-d8ef490456dc`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:33,005::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:19:33,006::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043102', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:33,006::task::1191::Storage.TaskManager.Task::(prepare) Task=`87eb570e-92cb-4033-b94f-d8ef490456dc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043102', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:33,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`87eb570e-92cb-4033-b94f-d8ef490456dc`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:33,006::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:33,007::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:33,007::task::993::Storage.TaskManager.Task::(_decref) Task=`87eb570e-92cb-4033-b94f-d8ef490456dc`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:19:33,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:33,532::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:33,533::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:19:33,583::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:19:33,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5b6ae07-7745-4a10-9178-a1cb15ea8e35`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:33,629::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:33,631::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:19:33,644::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:33,645::task::1191::Storage.TaskManager.Task::(prepare) Task=`e5b6ae07-7745-4a10-9178-a1cb15ea8e35`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:33,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5b6ae07-7745-4a10-9178-a1cb15ea8e35`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:33,648::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:33,649::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:33,650::task::993::Storage.TaskManager.Task::(_decref) Task=`e5b6ae07-7745-4a10-9178-a1cb15ea8e35`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:19:33,663::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:19:33,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`962d4125-d622-4337-9d32-a64b9a333822`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:19:33,669::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:19:33,673::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1eba8fa4-487c-427f-84b3-dd9589d18c74`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:19:33,674::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:19:33,676::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:19:33,677::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1eba8fa4-487c-427f-84b3-dd9589d18c74`::Granted request >Thread-34::DEBUG::2014-08-11 15:19:33,681::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`962d4125-d622-4337-9d32-a64b9a333822`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:19:33,682::task::993::Storage.TaskManager.Task::(_decref) Task=`962d4125-d622-4337-9d32-a64b9a333822`::ref 1 aborting False >Thread-14::DEBUG::2014-08-11 15:19:33,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:19:33,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:33,715::__init__::232::IOProcess::(_processLogs) (6770) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:19:33,719::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:19:33,747::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00344209 s, 132 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:19:33,753::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-34::INFO::2014-08-11 15:19:33,756::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854595530752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-14::DEBUG::2014-08-11 15:19:33,756::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-34::DEBUG::2014-08-11 15:19:33,758::task::1191::Storage.TaskManager.Task::(prepare) Task=`962d4125-d622-4337-9d32-a64b9a333822`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854595530752', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:33,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`962d4125-d622-4337-9d32-a64b9a333822`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:19:33,760::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-14::DEBUG::2014-08-11 15:19:33,760::__init__::232::IOProcess::(_processLogs) (6771) Got request for method 'statvfs' >Thread-34::DEBUG::2014-08-11 15:19:33,762::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:19:33,767::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-14::DEBUG::2014-08-11 15:19:33,767::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:19:33,770::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-34::DEBUG::2014-08-11 15:19:33,770::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-14::DEBUG::2014-08-11 15:19:33,771::__init__::232::IOProcess::(_processLogs) (6772) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 15:19:33,771::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:19:33,773::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-14::DEBUG::2014-08-11 15:19:33,773::__init__::232::IOProcess::(_processLogs) (6772) Queuing response >Thread-14::DEBUG::2014-08-11 15:19:33,776::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-34::DEBUG::2014-08-11 15:19:33,776::task::993::Storage.TaskManager.Task::(_decref) Task=`962d4125-d622-4337-9d32-a64b9a333822`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:19:33,779::__init__::232::IOProcess::(_processLogs) (6773) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:33,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:19:33,785::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:33,787::__init__::232::IOProcess::(_processLogs) (6774) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:19:33,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:33,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01321 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:35,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:35,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131644 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:37,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:38,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132549 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:38,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:38,641::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:38,725::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:19:40,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:40,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132651 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:42,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:42,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0138164 s, 74.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:43,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:43,729::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:19:43,791::__init__::232::IOProcess::(_processLogs) (6775) Got request for method 'statvfs' >Thread-115::DEBUG::2014-08-11 15:19:43,794::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-16::DEBUG::2014-08-11 15:19:43,829::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-115::DEBUG::2014-08-11 15:19:43,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ee48639-557b-4ce4-9dbe-8db6d63115d3`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:43,849::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:19:43,852::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 15:19:43,855::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000921512 s, 494 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:19:43,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:19:43,862::__init__::232::IOProcess::(_processLogs) (6776) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:19:43,864::__init__::232::IOProcess::(_processLogs) (6776) Queuing response >Thread-14::DEBUG::2014-08-11 15:19:43,869::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-115::INFO::2014-08-11 15:19:43,871::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:19:43,871::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-115::DEBUG::2014-08-11 15:19:43,873::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ee48639-557b-4ce4-9dbe-8db6d63115d3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:19:43,873::__init__::232::IOProcess::(_processLogs) (6777) Got request for method 'access' >Thread-115::DEBUG::2014-08-11 15:19:43,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ee48639-557b-4ce4-9dbe-8db6d63115d3`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:43,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:19:43,875::__init__::232::IOProcess::(_processLogs) (6777) Queuing response >Thread-115::DEBUG::2014-08-11 15:19:43,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 15:19:43,880::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-115::DEBUG::2014-08-11 15:19:43,879::task::993::Storage.TaskManager.Task::(_decref) Task=`1ee48639-557b-4ce4-9dbe-8db6d63115d3`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:19:43,882::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:43,884::__init__::232::IOProcess::(_processLogs) (6778) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:43,886::__init__::232::IOProcess::(_processLogs) (6778) Queuing response >Thread-14::DEBUG::2014-08-11 15:19:43,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:19:43,891::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:19:43,892::__init__::232::IOProcess::(_processLogs) (6779) Got request for method 'access' >Thread-34::DEBUG::2014-08-11 15:19:43,898::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:19:43,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`132c7180-d4bf-4c9e-8337-5aa25afe7262`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:19:43,901::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:19:43,904::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8538b1f-c950-4d2a-99e4-817fc0d5ec18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:19:43,905::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:19:43,906::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:19:43,907::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8538b1f-c950-4d2a-99e4-817fc0d5ec18`::Granted request >Thread-34::DEBUG::2014-08-11 15:19:43,910::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`132c7180-d4bf-4c9e-8337-5aa25afe7262`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:19:43,911::task::993::Storage.TaskManager.Task::(_decref) Task=`132c7180-d4bf-4c9e-8337-5aa25afe7262`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:19:43,972::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854497030144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:43,974::task::1191::Storage.TaskManager.Task::(prepare) Task=`132c7180-d4bf-4c9e-8337-5aa25afe7262`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854497030144', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:43,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`132c7180-d4bf-4c9e-8337-5aa25afe7262`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:19:43,976::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:19:43,977::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:19:43,979::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:19:43,980::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:19:43,981::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:19:43,983::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:19:43,985::task::993::Storage.TaskManager.Task::(_decref) Task=`132c7180-d4bf-4c9e-8337-5aa25afe7262`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:19:44,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:44,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132095 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:46,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:46,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131344 s, 78.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:19:48,119::task::595::Storage.TaskManager.Task::(_updateState) Task=`68f59072-1027-4232-9a80-f81784cc1ffd`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:48,119::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:19:48,120::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000921512', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:48,120::task::1191::Storage.TaskManager.Task::(prepare) Task=`68f59072-1027-4232-9a80-f81784cc1ffd`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000921512', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:19:48,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`68f59072-1027-4232-9a80-f81784cc1ffd`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:48,121::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:48,121::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:48,121::task::993::Storage.TaskManager.Task::(_decref) Task=`68f59072-1027-4232-9a80-f81784cc1ffd`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:19:48,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:19:48,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:48,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135452 s, 75.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:48,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:48,869::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:48,870::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:19:50,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:50,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133538 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:52,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:52,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133762 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:53,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:53,878::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:19:53,896::__init__::232::IOProcess::(_processLogs) (6780) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:53,927::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:19:53,929::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:19:53,948::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420763 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:19:53,950::__init__::232::IOProcess::(_processLogs) (6781) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:19:53,951::__init__::232::IOProcess::(_processLogs) (6782) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:53,952::__init__::232::IOProcess::(_processLogs) (6783) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:19:53,953::__init__::232::IOProcess::(_processLogs) (6784) Got request for method 'access' >Thread-115::DEBUG::2014-08-11 15:19:54,004::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:19:54,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bc37035-0979-4a6f-b460-d539df6d5264`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:19:54,006::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:19:54,018::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:54,019::task::1191::Storage.TaskManager.Task::(prepare) Task=`9bc37035-0979-4a6f-b460-d539df6d5264`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:19:54,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bc37035-0979-4a6f-b460-d539df6d5264`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:19:54,019::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:19:54,019::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:19:54,019::task::993::Storage.TaskManager.Task::(_decref) Task=`9bc37035-0979-4a6f-b460-d539df6d5264`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:19:54,033::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:19:54,034::task::595::Storage.TaskManager.Task::(_updateState) Task=`faf58fa2-6487-47c0-8212-50f76e56f5a2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:19:54,034::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:19:54,035::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a73fb82a-e45c-49df-881a-aacba5fb59b7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:19:54,035::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:19:54,036::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:19:54,036::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a73fb82a-e45c-49df-881a-aacba5fb59b7`::Granted request >Thread-34::DEBUG::2014-08-11 15:19:54,036::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`faf58fa2-6487-47c0-8212-50f76e56f5a2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:19:54,037::task::993::Storage.TaskManager.Task::(_decref) Task=`faf58fa2-6487-47c0-8212-50f76e56f5a2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:19:54,086::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854436278272', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:54,088::task::1191::Storage.TaskManager.Task::(prepare) Task=`faf58fa2-6487-47c0-8212-50f76e56f5a2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854436278272', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:19:54,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`faf58fa2-6487-47c0-8212-50f76e56f5a2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:19:54,090::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:19:54,091::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:19:54,093::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:19:54,095::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:19:54,096::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:19:54,097::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:19:54,099::task::993::Storage.TaskManager.Task::(_decref) Task=`faf58fa2-6487-47c0-8212-50f76e56f5a2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:19:54,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:54,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131726 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:56,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:56,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132901 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:19:58,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:19:58,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134565 s, 76.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:58,933::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:58,934::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:19:59,025::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:00,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:00,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133124 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:02,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:02,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0270367 s, 37.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:20:03,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d32b00f-503c-4e0f-9969-35d2d28b58a5`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:03,233::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:20:03,234::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420763', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:03,234::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d32b00f-503c-4e0f-9969-35d2d28b58a5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420763', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:03,234::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d32b00f-503c-4e0f-9969-35d2d28b58a5`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:03,234::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:03,234::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:03,235::task::993::Storage.TaskManager.Task::(_decref) Task=`7d32b00f-503c-4e0f-9969-35d2d28b58a5`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:20:03,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:20:03,955::__init__::232::IOProcess::(_processLogs) (6785) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:20:03,956::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:04,043::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:04,044::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:04,045::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:20:04,048::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416105 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:04,049::__init__::232::IOProcess::(_processLogs) (6786) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:20:04,050::__init__::232::IOProcess::(_processLogs) (6787) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:04,051::__init__::232::IOProcess::(_processLogs) (6788) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:04,052::__init__::232::IOProcess::(_processLogs) (6789) Got request for method 'access' >Thread-115::DEBUG::2014-08-11 15:20:04,123::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-115::DEBUG::2014-08-11 15:20:04,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c1263ba-6a7c-43cd-a7aa-046345ca4b7e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:04,125::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:20:04,137::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:04,138::task::1191::Storage.TaskManager.Task::(prepare) Task=`2c1263ba-6a7c-43cd-a7aa-046345ca4b7e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:04,138::task::595::Storage.TaskManager.Task::(_updateState) Task=`2c1263ba-6a7c-43cd-a7aa-046345ca4b7e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:04,138::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:04,138::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:04,138::task::993::Storage.TaskManager.Task::(_decref) Task=`2c1263ba-6a7c-43cd-a7aa-046345ca4b7e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:20:04,150::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 15:20:04,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8d11b01-2b1a-44a4-a9f6-1d7c3c6a6175`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:20:04,151::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:20:04,152::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a722314d-3501-45c8-bf7b-e3ae188d8a68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:20:04,152::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:20:04,152::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:20:04,152::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a722314d-3501-45c8-bf7b-e3ae188d8a68`::Granted request >Thread-34::DEBUG::2014-08-11 15:20:04,153::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d8d11b01-2b1a-44a4-a9f6-1d7c3c6a6175`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:20:04,153::task::993::Storage.TaskManager.Task::(_decref) Task=`d8d11b01-2b1a-44a4-a9f6-1d7c3c6a6175`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:20:04,174::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854358683648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:04,175::task::1191::Storage.TaskManager.Task::(prepare) Task=`d8d11b01-2b1a-44a4-a9f6-1d7c3c6a6175`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854358683648', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:04,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8d11b01-2b1a-44a4-a9f6-1d7c3c6a6175`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:20:04,178::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:20:04,179::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:20:04,181::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:20:04,184::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:20:04,185::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:20:04,187::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:20:04,190::task::993::Storage.TaskManager.Task::(_decref) Task=`d8d11b01-2b1a-44a4-a9f6-1d7c3c6a6175`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:20:04,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:04,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140038 s, 73.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:07,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:07,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131892 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:09,052::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:09,054::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:09,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:09,145::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:09,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133944 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:11,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:11,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132428 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:13,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:13,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013635 s, 75.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:14,055::__init__::232::IOProcess::(_processLogs) (6790) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:20:14,056::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:20:14,143::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00049344 s, 922 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:14,144::__init__::232::IOProcess::(_processLogs) (6791) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:20:14,145::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:20:14,146::__init__::232::IOProcess::(_processLogs) (6792) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:14,147::__init__::232::IOProcess::(_processLogs) (6793) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:14,148::__init__::232::IOProcess::(_processLogs) (6794) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:14,153::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:14,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:14,155::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:20:14,206::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:20:14,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a307a0b-bcb5-4734-a2bd-1c38011dedc9`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:14,208::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:20:14,220::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:14,221::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a307a0b-bcb5-4734-a2bd-1c38011dedc9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:14,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a307a0b-bcb5-4734-a2bd-1c38011dedc9`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:14,221::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:14,221::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:14,222::task::993::Storage.TaskManager.Task::(_decref) Task=`4a307a0b-bcb5-4734-a2bd-1c38011dedc9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:20:14,232::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:20:14,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7bdc338-95b8-441a-8053-18c3b801f52f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:20:14,233::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:20:14,234::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`91462145-cee2-4e35-bc45-6350877a603d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:20:14,234::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:20:14,235::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:20:14,235::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`91462145-cee2-4e35-bc45-6350877a603d`::Granted request >Thread-34::DEBUG::2014-08-11 15:20:14,235::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f7bdc338-95b8-441a-8053-18c3b801f52f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:20:14,236::task::993::Storage.TaskManager.Task::(_decref) Task=`f7bdc338-95b8-441a-8053-18c3b801f52f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:20:14,255::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854266540032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:14,256::task::1191::Storage.TaskManager.Task::(prepare) Task=`f7bdc338-95b8-441a-8053-18c3b801f52f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854266540032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:14,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7bdc338-95b8-441a-8053-18c3b801f52f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:20:14,256::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:20:14,256::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:20:14,257::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:20:14,257::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:20:14,257::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:20:14,257::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:20:14,258::task::993::Storage.TaskManager.Task::(_decref) Task=`f7bdc338-95b8-441a-8053-18c3b801f52f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:20:15,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:15,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132065 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:17,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:17,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132306 s, 77.4 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:20:17,956::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e4f58da-0c66-485f-bdc2-70249ad8d8d4`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:20:17,956::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:20:17,957::__init__::232::IOProcess::(_processLogs) (6795) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:20:17,966::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:20:17,967::__init__::232::IOProcess::(_processLogs) (6796) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:20:17,967::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:20:17,967::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e4f58da-0c66-485f-bdc2-70249ad8d8d4`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:20:17,968::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e4f58da-0c66-485f-bdc2-70249ad8d8d4`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:20:17,968::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:20:17,968::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:20:17,968::task::993::Storage.TaskManager.Task::(_decref) Task=`2e4f58da-0c66-485f-bdc2-70249ad8d8d4`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:20:18,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cab1411-6ef8-4f4d-a1b6-bec8fad83395`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:18,356::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:20:18,356::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00049344', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:18,357::task::1191::Storage.TaskManager.Task::(prepare) Task=`9cab1411-6ef8-4f4d-a1b6-bec8fad83395`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00049344', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:18,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cab1411-6ef8-4f4d-a1b6-bec8fad83395`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:18,357::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:18,357::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:18,358::task::993::Storage.TaskManager.Task::(_decref) Task=`9cab1411-6ef8-4f4d-a1b6-bec8fad83395`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:20:18,376::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:19,161::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:19,163::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:19,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:19,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:19,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132692 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:21,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:21,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133637 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:23,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:23,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131695 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:24,150::__init__::232::IOProcess::(_processLogs) (6797) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:20:24,151::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:20:24,170::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426039 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:24,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:20:24,224::__init__::232::IOProcess::(_processLogs) (6798) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:20:24,226::__init__::232::IOProcess::(_processLogs) (6799) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:24,227::__init__::232::IOProcess::(_processLogs) (6800) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:24,228::__init__::232::IOProcess::(_processLogs) (6801) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:24,230::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:24,232::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:24,233::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:20:24,271::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:20:24,272::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e8cf815-6aa4-4667-a7f6-b916df99b86d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:24,272::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:20:24,285::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:24,286::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e8cf815-6aa4-4667-a7f6-b916df99b86d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:24,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e8cf815-6aa4-4667-a7f6-b916df99b86d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:24,286::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:24,286::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:24,286::task::993::Storage.TaskManager.Task::(_decref) Task=`6e8cf815-6aa4-4667-a7f6-b916df99b86d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:20:24,296::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:20:24,297::task::595::Storage.TaskManager.Task::(_updateState) Task=`2efa46f9-9d4f-4b25-83d8-fa8679aa46ac`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:20:24,297::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:20:24,300::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7e1c1e2-4f0c-4e14-adf8-828c5ffcafab`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:20:24,301::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:20:24,301::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:20:24,301::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7e1c1e2-4f0c-4e14-adf8-828c5ffcafab`::Granted request >Thread-34::DEBUG::2014-08-11 15:20:24,302::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2efa46f9-9d4f-4b25-83d8-fa8679aa46ac`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:20:24,302::task::993::Storage.TaskManager.Task::(_decref) Task=`2efa46f9-9d4f-4b25-83d8-fa8679aa46ac`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:20:24,322::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854178787328', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:24,322::task::1191::Storage.TaskManager.Task::(prepare) Task=`2efa46f9-9d4f-4b25-83d8-fa8679aa46ac`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854178787328', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:24,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`2efa46f9-9d4f-4b25-83d8-fa8679aa46ac`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:20:24,323::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:20:24,323::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:20:24,323::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:20:24,323::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:20:24,324::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:20:24,324::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:20:24,324::task::993::Storage.TaskManager.Task::(_decref) Task=`2efa46f9-9d4f-4b25-83d8-fa8679aa46ac`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:20:25,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:25,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133555 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:27,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:27,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132331 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:29,238::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:29,239::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:29,325::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:29,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:29,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013214 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:31,742::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:31,776::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134062 s, 76.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:20:33,522::task::595::Storage.TaskManager.Task::(_updateState) Task=`1dcd1237-0de5-4a86-bc0b-0eec6ee9b3ff`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:33,522::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:20:33,523::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426039', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:33,523::task::1191::Storage.TaskManager.Task::(prepare) Task=`1dcd1237-0de5-4a86-bc0b-0eec6ee9b3ff`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426039', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:33,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`1dcd1237-0de5-4a86-bc0b-0eec6ee9b3ff`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:33,523::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:33,523::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:33,524::task::993::Storage.TaskManager.Task::(_decref) Task=`1dcd1237-0de5-4a86-bc0b-0eec6ee9b3ff`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:20:33,547::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:33,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:33,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131674 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:34,230::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:20:34,230::__init__::232::IOProcess::(_processLogs) (6802) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:20:34,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:20:34,251::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000651907 s, 698 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:34,252::__init__::232::IOProcess::(_processLogs) (6803) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:20:34,326::__init__::232::IOProcess::(_processLogs) (6804) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:34,327::__init__::232::IOProcess::(_processLogs) (6805) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:34,328::__init__::232::IOProcess::(_processLogs) (6806) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:34,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:34,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:34,335::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:20:34,344::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:20:34,344::task::595::Storage.TaskManager.Task::(_updateState) Task=`6524a511-0b1e-479b-97ac-3effdd65c74b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:34,345::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:20:34,357::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:34,358::task::1191::Storage.TaskManager.Task::(prepare) Task=`6524a511-0b1e-479b-97ac-3effdd65c74b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:34,358::task::595::Storage.TaskManager.Task::(_updateState) Task=`6524a511-0b1e-479b-97ac-3effdd65c74b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:34,358::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:34,358::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:34,359::task::993::Storage.TaskManager.Task::(_decref) Task=`6524a511-0b1e-479b-97ac-3effdd65c74b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:20:34,369::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:20:34,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`626d83fb-eb3b-403b-b206-89439b65205e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:20:34,370::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:20:34,370::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b32aaad5-b578-4214-adf1-67f00c8f8f21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:20:34,371::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:20:34,371::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:20:34,371::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b32aaad5-b578-4214-adf1-67f00c8f8f21`::Granted request >Thread-34::DEBUG::2014-08-11 15:20:34,372::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`626d83fb-eb3b-403b-b206-89439b65205e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:20:34,372::task::993::Storage.TaskManager.Task::(_decref) Task=`626d83fb-eb3b-403b-b206-89439b65205e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:20:34,392::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854069604352', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:34,392::task::1191::Storage.TaskManager.Task::(prepare) Task=`626d83fb-eb3b-403b-b206-89439b65205e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1854069604352', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:34,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`626d83fb-eb3b-403b-b206-89439b65205e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:20:34,393::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:20:34,393::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:20:34,393::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:20:34,393::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:20:34,394::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:20:34,394::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:20:34,394::task::993::Storage.TaskManager.Task::(_decref) Task=`626d83fb-eb3b-403b-b206-89439b65205e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:20:35,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:35,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132454 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:37,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:37,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132326 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:39,338::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:39,339::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:39,340::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:39,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:39,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132694 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:42,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:42,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132468 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:44,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:44,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.014006 s, 73.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:44,330::__init__::232::IOProcess::(_processLogs) (6807) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:20:44,331::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:44,349::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:20:44,352::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00209057 s, 218 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:44,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:20:44,406::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:20:44,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d1a140c-c664-4a47-8c7d-a4707831b90a`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:20:44,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-115::INFO::2014-08-11 15:20:44,430::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:20:44,432::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:44,432::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:20:44,433::__init__::232::IOProcess::(_processLogs) (6808) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:20:44,436::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:20:44,437::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:20:44,438::__init__::232::IOProcess::(_processLogs) (6809) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:44,439::__init__::232::IOProcess::(_processLogs) (6809) Queuing response >Thread-14::DEBUG::2014-08-11 15:20:44,442::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:20:44,442::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:20:44,443::__init__::232::IOProcess::(_processLogs) (6810) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:44,443::__init__::232::IOProcess::(_processLogs) (6810) Queuing response >Thread-14::DEBUG::2014-08-11 15:20:44,443::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:20:44,444::__init__::232::IOProcess::(_processLogs) (6811) Got request for method 'access' >Thread-115::INFO::2014-08-11 15:20:44,445::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:44,446::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d1a140c-c664-4a47-8c7d-a4707831b90a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:44,446::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d1a140c-c664-4a47-8c7d-a4707831b90a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:44,446::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:44,447::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:44,447::task::993::Storage.TaskManager.Task::(_decref) Task=`8d1a140c-c664-4a47-8c7d-a4707831b90a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:20:44,459::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:20:44,459::task::595::Storage.TaskManager.Task::(_updateState) Task=`506208de-97b6-41f3-ae67-1f6823cd0654`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:20:44,460::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:20:44,460::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a1ef0a74-3ebd-4a29-807d-9ccba97c9767`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:20:44,461::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:20:44,461::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:20:44,461::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a1ef0a74-3ebd-4a29-807d-9ccba97c9767`::Granted request >Thread-34::DEBUG::2014-08-11 15:20:44,462::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`506208de-97b6-41f3-ae67-1f6823cd0654`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:20:44,462::task::993::Storage.TaskManager.Task::(_decref) Task=`506208de-97b6-41f3-ae67-1f6823cd0654`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:20:44,483::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1853752016896', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:44,483::task::1191::Storage.TaskManager.Task::(prepare) Task=`506208de-97b6-41f3-ae67-1f6823cd0654`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1853752016896', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:44,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`506208de-97b6-41f3-ae67-1f6823cd0654`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:20:44,484::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:20:44,484::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:20:44,484::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:20:44,485::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:20:44,485::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:20:44,485::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:20:44,485::task::993::Storage.TaskManager.Task::(_decref) Task=`506208de-97b6-41f3-ae67-1f6823cd0654`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:20:46,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:46,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132181 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:48,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:48,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013904 s, 73.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:20:48,633::task::595::Storage.TaskManager.Task::(_updateState) Task=`5b0ebe45-b539-47d4-af4c-9b4db541e739`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:48,633::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:20:48,633::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00209057', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:48,634::task::1191::Storage.TaskManager.Task::(prepare) Task=`5b0ebe45-b539-47d4-af4c-9b4db541e739`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00209057', 'lastCheck': '4.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:20:48,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`5b0ebe45-b539-47d4-af4c-9b4db541e739`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:48,634::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:48,634::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:48,635::task::993::Storage.TaskManager.Task::(_decref) Task=`5b0ebe45-b539-47d4-af4c-9b4db541e739`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:20:48,654::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:49,448::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:49,449::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:49,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:20:50,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:50,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131157 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:52,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:52,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013068 s, 78.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:20:54,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:54,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135228 s, 75.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:54,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:20:54,447::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:20:54,448::__init__::232::IOProcess::(_processLogs) (6812) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:20:54,448::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:20:54,467::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420568 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:20:54,469::__init__::232::IOProcess::(_processLogs) (6813) Got request for method 'statvfs' >Thread-115::DEBUG::2014-08-11 15:20:54,500::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:20:54,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`21f4e047-b26e-48ca-b286-b46d5c48623b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:20:54,527::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:20:54,528::__init__::232::IOProcess::(_processLogs) (6814) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:54,531::__init__::232::IOProcess::(_processLogs) (6815) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:54,532::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:20:54,533::__init__::232::IOProcess::(_processLogs) (6815) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:54,535::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:54,539::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:20:54,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:20:54,542::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:20:54,543::__init__::232::IOProcess::(_processLogs) (6816) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:20:54,543::__init__::232::IOProcess::(_processLogs) (6816) Queuing response >Detector thread::DEBUG::2014-08-11 15:20:55,158::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:36399 >Detector thread::DEBUG::2014-08-11 15:20:55,158::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:36399 >Detector thread::DEBUG::2014-08-11 15:20:55,158::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:36399 >Detector thread::DEBUG::2014-08-11 15:20:55,159::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 36399) >Thread-115::INFO::2014-08-11 15:20:56,391::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:56,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`21f4e047-b26e-48ca-b286-b46d5c48623b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:20:56,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`21f4e047-b26e-48ca-b286-b46d5c48623b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:20:56,392::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:20:56,392::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:20:56,392::task::993::Storage.TaskManager.Task::(_decref) Task=`21f4e047-b26e-48ca-b286-b46d5c48623b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:20:56,404::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:20:56,405::task::595::Storage.TaskManager.Task::(_updateState) Task=`84cc0e85-862b-422c-bdf7-007433c938ab`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:20:56,405::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:20:56,406::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f9bad49f-6c24-4a5e-9cdf-4e51baa5a164`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:20:56,406::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:20:56,407::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:20:56,407::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f9bad49f-6c24-4a5e-9cdf-4e51baa5a164`::Granted request >Thread-34::DEBUG::2014-08-11 15:20:56,407::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`84cc0e85-862b-422c-bdf7-007433c938ab`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:20:56,408::task::993::Storage.TaskManager.Task::(_decref) Task=`84cc0e85-862b-422c-bdf7-007433c938ab`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 15:20:56,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:56,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140506 s, 72.9 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 15:20:56,457::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1853493018624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:56,458::task::1191::Storage.TaskManager.Task::(prepare) Task=`84cc0e85-862b-422c-bdf7-007433c938ab`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1853493018624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:20:56,459::task::595::Storage.TaskManager.Task::(_updateState) Task=`84cc0e85-862b-422c-bdf7-007433c938ab`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:20:56,460::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:20:56,461::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:20:56,463::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:20:56,465::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:20:56,466::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:20:56,467::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:20:56,468::task::993::Storage.TaskManager.Task::(_decref) Task=`84cc0e85-862b-422c-bdf7-007433c938ab`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:20:58,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:20:58,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133302 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:59,548::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:59,550::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:20:59,625::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:00,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:00,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013381 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:02,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:02,647::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267334 s, 38.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:04,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`f95eb475-3b3b-431d-9286-4dc62b1105c0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:04,205::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:21:04,205::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420568', 'lastCheck': '9.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:04,206::task::1191::Storage.TaskManager.Task::(prepare) Task=`f95eb475-3b3b-431d-9286-4dc62b1105c0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420568', 'lastCheck': '9.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:04,206::task::595::Storage.TaskManager.Task::(_updateState) Task=`f95eb475-3b3b-431d-9286-4dc62b1105c0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:04,206::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:04,206::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:04,207::task::993::Storage.TaskManager.Task::(_decref) Task=`f95eb475-3b3b-431d-9286-4dc62b1105c0`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:21:04,228::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:04,545::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:04,545::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:04,545::__init__::232::IOProcess::(_processLogs) (6817) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:04,546::__init__::232::IOProcess::(_processLogs) DEBUG|(6817) Queuing response >Thread-16::DEBUG::2014-08-11 15:21:04,547::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:21:04,567::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000435942 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:04,569::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:04,632::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:04,727::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:04,731::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:04,733::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:04,734::__init__::232::IOProcess::(_processLogs) (6818) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:04,737::__init__::232::IOProcess::(_processLogs) (6818) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:04,739::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:04,741::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:04,744::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:04,745::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:04,747::__init__::232::IOProcess::(_processLogs) (6819) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:04,749::__init__::232::IOProcess::(_processLogs) DEBUG|(6819) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:04,750::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:04,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:04,753::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:04,754::__init__::232::IOProcess::(_processLogs) (6820) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:04,756::__init__::232::IOProcess::(_processLogs) DEBUG|(6820) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:04,757::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:04,758::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:04,759::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:04,761::__init__::232::IOProcess::(_processLogs) (6821) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:04,762::__init__::232::IOProcess::(_processLogs) (6821) Queuing response >Dummy-33::DEBUG::2014-08-11 15:21:04,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:04,869::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133687 s, 76.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:06,484::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:21:06,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dee604f-1b24-430c-b98a-26930968b9fd`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:06,485::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:21:06,498::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:06,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`0dee604f-1b24-430c-b98a-26930968b9fd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:06,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dee604f-1b24-430c-b98a-26930968b9fd`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:06,499::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:06,499::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:06,499::task::993::Storage.TaskManager.Task::(_decref) Task=`0dee604f-1b24-430c-b98a-26930968b9fd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:21:06,510::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:21:06,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc570768-19d8-46e1-aea7-e60b955bf15d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:21:06,512::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:21:06,512::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce716dbc-ab59-48de-8514-e5193bdc063f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:21:06,513::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:21:06,513::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:21:06,513::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ce716dbc-ab59-48de-8514-e5193bdc063f`::Granted request >Thread-34::DEBUG::2014-08-11 15:21:06,514::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cc570768-19d8-46e1-aea7-e60b955bf15d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:21:06,514::task::993::Storage.TaskManager.Task::(_decref) Task=`cc570768-19d8-46e1-aea7-e60b955bf15d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:21:06,534::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1853249224704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:06,534::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc570768-19d8-46e1-aea7-e60b955bf15d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1853249224704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:06,534::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc570768-19d8-46e1-aea7-e60b955bf15d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:21:06,535::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:21:06,535::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:21:06,535::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:21:06,535::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:21:06,536::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:21:06,536::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:21:06,536::task::993::Storage.TaskManager.Task::(_decref) Task=`cc570768-19d8-46e1-aea7-e60b955bf15d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:21:06,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:06,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132921 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:08,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:08,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132653 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:09,747::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:09,748::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:09,827::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:11,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:11,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139129 s, 73.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:13,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:13,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132579 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:14,761::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:14,827::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:21:14,828::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:21:14,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:14,847::__init__::232::IOProcess::(_processLogs) (6822) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:14,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:21:14,851::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000573371 s, 794 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:14,852::__init__::232::IOProcess::(_processLogs) (6822) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:14,856::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:14,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:14,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:14,859::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:14,860::__init__::232::IOProcess::(_processLogs) (6823) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:14,863::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:14,863::__init__::232::IOProcess::(_processLogs) DEBUG|(6823) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:14,867::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:14,870::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:14,871::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:14,872::__init__::232::IOProcess::(_processLogs) (6824) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:14,874::__init__::232::IOProcess::(_processLogs) (6824) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:14,876::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:14,877::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:14,878::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:14,879::__init__::232::IOProcess::(_processLogs) (6825) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:14,880::__init__::232::IOProcess::(_processLogs) DEBUG|(6825) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:14,882::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:14,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:14,885::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:14,886::__init__::232::IOProcess::(_processLogs) (6826) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:14,889::__init__::232::IOProcess::(_processLogs) (6826) Queuing response >Dummy-33::DEBUG::2014-08-11 15:21:15,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:15,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140235 s, 73.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:16,548::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:21:16,549::task::595::Storage.TaskManager.Task::(_updateState) Task=`12ad3f30-0f96-4f7c-bff6-d06a8dbe8df8`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:16,549::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:21:16,562::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:16,562::task::1191::Storage.TaskManager.Task::(prepare) Task=`12ad3f30-0f96-4f7c-bff6-d06a8dbe8df8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:16,563::task::595::Storage.TaskManager.Task::(_updateState) Task=`12ad3f30-0f96-4f7c-bff6-d06a8dbe8df8`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:16,563::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:16,563::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:16,563::task::993::Storage.TaskManager.Task::(_decref) Task=`12ad3f30-0f96-4f7c-bff6-d06a8dbe8df8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:21:16,574::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:21:16,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`97f0f678-89f1-468b-bf01-4035e33fe2fe`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:21:16,574::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:21:16,575::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8e7abcda-5b40-451d-a397-67dc8683ade4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:21:16,575::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:21:16,576::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:21:16,576::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8e7abcda-5b40-451d-a397-67dc8683ade4`::Granted request >Thread-34::DEBUG::2014-08-11 15:21:16,577::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`97f0f678-89f1-468b-bf01-4035e33fe2fe`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:21:16,577::task::993::Storage.TaskManager.Task::(_decref) Task=`97f0f678-89f1-468b-bf01-4035e33fe2fe`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:21:16,597::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852991012864', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:16,597::task::1191::Storage.TaskManager.Task::(prepare) Task=`97f0f678-89f1-468b-bf01-4035e33fe2fe`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852991012864', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:16,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`97f0f678-89f1-468b-bf01-4035e33fe2fe`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:21:16,598::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:21:16,598::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:21:16,598::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:21:16,599::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:21:16,599::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:21:16,599::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:21:16,599::task::993::Storage.TaskManager.Task::(_decref) Task=`97f0f678-89f1-468b-bf01-4035e33fe2fe`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:21:17,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:17,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131374 s, 77.9 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:21:18,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`eac31851-38a6-44f2-9869-3a5543c11d68`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:21:18,076::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:21:18,077::__init__::232::IOProcess::(_processLogs) (6827) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:21:18,079::__init__::232::IOProcess::(_processLogs) (6828) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:21:18,080::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:21:18,080::task::1191::Storage.TaskManager.Task::(prepare) Task=`eac31851-38a6-44f2-9869-3a5543c11d68`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:21:18,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`eac31851-38a6-44f2-9869-3a5543c11d68`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:21:18,081::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:21:18,081::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:21:18,081::task::993::Storage.TaskManager.Task::(_decref) Task=`eac31851-38a6-44f2-9869-3a5543c11d68`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:21:19,247::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:19,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135926 s, 75.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:19,347::task::595::Storage.TaskManager.Task::(_updateState) Task=`608421e1-03db-4d23-96ec-13075d8a8685`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:19,348::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:21:19,348::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000573371', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:19,349::task::1191::Storage.TaskManager.Task::(prepare) Task=`608421e1-03db-4d23-96ec-13075d8a8685`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000573371', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:19,349::task::595::Storage.TaskManager.Task::(_updateState) Task=`608421e1-03db-4d23-96ec-13075d8a8685`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:19,349::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:19,349::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:19,349::task::993::Storage.TaskManager.Task::(_decref) Task=`608421e1-03db-4d23-96ec-13075d8a8685`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:21:19,369::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:19,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:19,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:19,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:21,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:21,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133173 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:23,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:23,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132533 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:24,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:24,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:24,887::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:24,890::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:24,890::__init__::232::IOProcess::(_processLogs) (6829) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:21:24,891::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:21:24,911::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425559 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:24,912::__init__::232::IOProcess::(_processLogs) (6830) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:24,914::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:24,914::__init__::232::IOProcess::(_processLogs) (6831) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:24,915::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:24,915::__init__::232::IOProcess::(_processLogs) (6832) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:24,916::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:24,916::__init__::232::IOProcess::(_processLogs) (6833) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:21:25,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:25,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133233 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:26,612::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:21:26,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc48f86c-2d6a-4925-8c24-7b8acf1ec253`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:26,613::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:21:26,626::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:26,627::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc48f86c-2d6a-4925-8c24-7b8acf1ec253`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:26,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc48f86c-2d6a-4925-8c24-7b8acf1ec253`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:26,627::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:26,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:26,628::task::993::Storage.TaskManager.Task::(_decref) Task=`cc48f86c-2d6a-4925-8c24-7b8acf1ec253`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:21:26,638::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:21:26,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`28877d5f-c22a-4b65-ad5f-6bb0d423b483`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:21:26,639::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:21:26,639::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0e7cc400-8af2-47db-9d0f-69f91be77033`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:21:26,640::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:21:26,640::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:21:26,640::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0e7cc400-8af2-47db-9d0f-69f91be77033`::Granted request >Thread-34::DEBUG::2014-08-11 15:21:26,641::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`28877d5f-c22a-4b65-ad5f-6bb0d423b483`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:21:26,641::task::993::Storage.TaskManager.Task::(_decref) Task=`28877d5f-c22a-4b65-ad5f-6bb0d423b483`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:21:26,661::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:26,661::task::1191::Storage.TaskManager.Task::(prepare) Task=`28877d5f-c22a-4b65-ad5f-6bb0d423b483`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:26,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`28877d5f-c22a-4b65-ad5f-6bb0d423b483`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:21:26,662::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:21:26,662::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:21:26,662::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:21:26,663::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:21:26,663::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:21:26,663::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:21:26,663::task::993::Storage.TaskManager.Task::(_decref) Task=`28877d5f-c22a-4b65-ad5f-6bb0d423b483`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:21:27,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:27,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132224 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:29,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:29,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132702 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:29,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:29,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:29,927::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:31,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:31,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133307 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:33,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:33,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132622 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:34,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`f72095ea-3761-4c3d-bba3-33e399e0b148`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:34,476::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:21:34,476::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425559', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:34,476::task::1191::Storage.TaskManager.Task::(prepare) Task=`f72095ea-3761-4c3d-bba3-33e399e0b148`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425559', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:34,477::task::595::Storage.TaskManager.Task::(_updateState) Task=`f72095ea-3761-4c3d-bba3-33e399e0b148`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:34,477::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:34,477::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:34,477::task::993::Storage.TaskManager.Task::(_decref) Task=`f72095ea-3761-4c3d-bba3-33e399e0b148`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:21:34,503::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:34,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:34,919::__init__::232::IOProcess::(_processLogs) (6834) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:21:34,920::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:21:34,927::__init__::232::IOProcess::(_processLogs) (6834) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:34,947::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:34,949::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:21:34,950::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000405275 s, 1.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:34,953::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:34,954::__init__::232::IOProcess::(_processLogs) (6835) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:34,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:34,956::__init__::232::IOProcess::(_processLogs) (6836) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:34,957::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:34,957::__init__::232::IOProcess::(_processLogs) (6837) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:34,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:34,958::__init__::232::IOProcess::(_processLogs) (6838) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:34,959::__init__::232::IOProcess::(_processLogs) (6838) Queuing response >Dummy-33::DEBUG::2014-08-11 15:21:35,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:35,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013206 s, 77.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:36,675::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:21:36,676::task::595::Storage.TaskManager.Task::(_updateState) Task=`e60f708c-8a3c-450d-bc4c-2f897d567fa4`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:36,676::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:21:36,689::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:36,689::task::1191::Storage.TaskManager.Task::(prepare) Task=`e60f708c-8a3c-450d-bc4c-2f897d567fa4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:36,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`e60f708c-8a3c-450d-bc4c-2f897d567fa4`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:36,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:36,690::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:36,690::task::993::Storage.TaskManager.Task::(_decref) Task=`e60f708c-8a3c-450d-bc4c-2f897d567fa4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:21:36,702::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:21:36,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca1b52af-995f-4156-9ce4-e6a3a7f87c9a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:21:36,703::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:21:36,704::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30b4a4d2-061e-412f-851c-aac629ea896d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:21:36,704::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:21:36,704::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:21:36,704::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30b4a4d2-061e-412f-851c-aac629ea896d`::Granted request >Thread-34::DEBUG::2014-08-11 15:21:36,705::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ca1b52af-995f-4156-9ce4-e6a3a7f87c9a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:21:36,705::task::993::Storage.TaskManager.Task::(_decref) Task=`ca1b52af-995f-4156-9ce4-e6a3a7f87c9a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:21:36,725::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:36,726::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca1b52af-995f-4156-9ce4-e6a3a7f87c9a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:36,726::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca1b52af-995f-4156-9ce4-e6a3a7f87c9a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:21:36,726::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:21:36,726::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:21:36,727::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:21:36,727::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:21:36,727::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:21:36,727::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:21:36,728::task::993::Storage.TaskManager.Task::(_decref) Task=`ca1b52af-995f-4156-9ce4-e6a3a7f87c9a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:21:37,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:37,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140854 s, 72.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:39,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:39,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131579 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:39,961::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:39,962::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:39,963::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:41,860::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:41,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131569 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:43,916::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:43,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132174 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:44,961::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:44,962::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:21:44,963::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:44,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:45,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:45,047::__init__::232::IOProcess::(_processLogs) (6839) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:45,048::__init__::232::IOProcess::(_processLogs) (6839) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:45,050::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:45,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:21:45,053::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000526159 s, 865 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:45,054::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:45,054::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:45,055::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:45,055::__init__::232::IOProcess::(_processLogs) (6840) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:45,055::__init__::232::IOProcess::(_processLogs) (6840) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:45,056::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:45,057::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:45,057::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:45,058::__init__::232::IOProcess::(_processLogs) (6841) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:45,058::__init__::232::IOProcess::(_processLogs) (6841) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:45,058::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:45,059::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:45,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:45,060::__init__::232::IOProcess::(_processLogs) (6842) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:45,060::__init__::232::IOProcess::(_processLogs) (6842) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:45,060::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:45,061::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:45,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:45,062::__init__::232::IOProcess::(_processLogs) (6843) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:45,063::__init__::232::IOProcess::(_processLogs) (6843) Queuing response >Dummy-33::DEBUG::2014-08-11 15:21:45,972::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:46,005::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132435 s, 77.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:46,742::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:21:46,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`b06e2391-38fa-4e2e-af18-112177a4b77c`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:46,743::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:21:46,756::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:46,756::task::1191::Storage.TaskManager.Task::(prepare) Task=`b06e2391-38fa-4e2e-af18-112177a4b77c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:46,756::task::595::Storage.TaskManager.Task::(_updateState) Task=`b06e2391-38fa-4e2e-af18-112177a4b77c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:46,756::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:46,757::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:46,757::task::993::Storage.TaskManager.Task::(_decref) Task=`b06e2391-38fa-4e2e-af18-112177a4b77c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:21:46,768::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:21:46,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`04085b5c-b9bc-4b36-a3fa-c55ba41ca916`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:21:46,769::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:21:46,770::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`59110ebe-b86c-47ba-a6c6-76b60a73e238`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:21:46,770::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:21:46,771::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:21:46,771::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`59110ebe-b86c-47ba-a6c6-76b60a73e238`::Granted request >Thread-34::DEBUG::2014-08-11 15:21:46,771::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`04085b5c-b9bc-4b36-a3fa-c55ba41ca916`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:21:46,772::task::993::Storage.TaskManager.Task::(_decref) Task=`04085b5c-b9bc-4b36-a3fa-c55ba41ca916`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:21:46,793::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:46,793::task::1191::Storage.TaskManager.Task::(prepare) Task=`04085b5c-b9bc-4b36-a3fa-c55ba41ca916`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:46,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`04085b5c-b9bc-4b36-a3fa-c55ba41ca916`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:21:46,794::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:21:46,794::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:21:46,794::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:21:46,795::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:21:46,795::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:21:46,795::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:21:46,795::task::993::Storage.TaskManager.Task::(_decref) Task=`04085b5c-b9bc-4b36-a3fa-c55ba41ca916`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:21:48,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:48,061::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132667 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:49,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d265ecb-d684-480e-8e52-2e418a158f4e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:49,599::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:21:49,599::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000526159', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:49,600::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d265ecb-d684-480e-8e52-2e418a158f4e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000526159', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:21:49,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d265ecb-d684-480e-8e52-2e418a158f4e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:49,600::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:49,600::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:49,601::task::993::Storage.TaskManager.Task::(_decref) Task=`6d265ecb-d684-480e-8e52-2e418a158f4e`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:21:49,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:50,057::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:50,058::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:50,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:50,148::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:50,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136405 s, 75.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:52,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:52,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132373 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:21:54,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:54,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131651 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:55,063::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:55,064::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:21:55,065::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:21:55,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:55,147::__init__::232::IOProcess::(_processLogs) (6844) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:55,148::__init__::232::IOProcess::(_processLogs) (6844) Queuing response >Thread-16::DEBUG::2014-08-11 15:21:55,149::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000565128 s, 805 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:21:55,150::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:21:55,150::__init__::232::IOProcess::(_processLogs) (6838) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:55,150::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:55,151::__init__::232::IOProcess::(_processLogs) (6845) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:21:55,152::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:55,152::__init__::232::IOProcess::(_processLogs) (6846) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:55,153::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:21:55,153::__init__::232::IOProcess::(_processLogs) (6847) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:21:55,154::__init__::232::IOProcess::(_processLogs) (6847) Queuing response >Thread-14::DEBUG::2014-08-11 15:21:55,155::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:21:55,157::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:55,157::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:21:55,159::__init__::232::IOProcess::(_processLogs) (6848) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:55,163::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:21:55,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:21:56,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:56,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133366 s, 76.8 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:21:56,806::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:21:56,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ab850ce-fa4e-41c4-9bdf-0dd55fdd62ad`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:21:56,808::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:21:56,820::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:56,821::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ab850ce-fa4e-41c4-9bdf-0dd55fdd62ad`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:21:56,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ab850ce-fa4e-41c4-9bdf-0dd55fdd62ad`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:21:56,821::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:21:56,822::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:21:56,822::task::993::Storage.TaskManager.Task::(_decref) Task=`9ab850ce-fa4e-41c4-9bdf-0dd55fdd62ad`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:21:56,832::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:21:56,833::task::595::Storage.TaskManager.Task::(_updateState) Task=`b91cfba9-f8e2-458e-881a-b60ab3959239`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:21:56,833::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:21:56,834::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7f59a533-d2b6-473e-b715-0b473a454850`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:21:56,834::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:21:56,835::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:21:56,835::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7f59a533-d2b6-473e-b715-0b473a454850`::Granted request >Thread-34::DEBUG::2014-08-11 15:21:56,836::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b91cfba9-f8e2-458e-881a-b60ab3959239`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:21:56,836::task::993::Storage.TaskManager.Task::(_decref) Task=`b91cfba9-f8e2-458e-881a-b60ab3959239`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:21:56,856::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:56,856::task::1191::Storage.TaskManager.Task::(prepare) Task=`b91cfba9-f8e2-458e-881a-b60ab3959239`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959555584', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:21:56,856::task::595::Storage.TaskManager.Task::(_updateState) Task=`b91cfba9-f8e2-458e-881a-b60ab3959239`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:21:56,856::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:21:56,857::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:21:56,857::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:21:56,857::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:21:56,858::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:21:56,858::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:21:56,858::task::993::Storage.TaskManager.Task::(_decref) Task=`b91cfba9-f8e2-458e-881a-b60ab3959239`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:21:58,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:21:58,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132245 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:00,167::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:00,169::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:00,229::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:00,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:00,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134517 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:02,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:02,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278228 s, 36.8 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:04,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2a7b528-4548-4d36-bdba-2833e63806cb`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:04,711::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:22:04,711::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000565128', 'lastCheck': '9.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:04,712::task::1191::Storage.TaskManager.Task::(prepare) Task=`e2a7b528-4548-4d36-bdba-2833e63806cb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000565128', 'lastCheck': '9.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:04,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`e2a7b528-4548-4d36-bdba-2833e63806cb`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:04,712::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Dummy-33::DEBUG::2014-08-11 15:22:04,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-115::DEBUG::2014-08-11 15:22:04,714::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:04,732::task::993::Storage.TaskManager.Task::(_decref) Task=`e2a7b528-4548-4d36-bdba-2833e63806cb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:04,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132858 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:04,762::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:22:05,164::__init__::232::IOProcess::(_processLogs) (6849) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:22:05,165::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:22:05,183::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00043271 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:05,185::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:05,228::__init__::232::IOProcess::(_processLogs) (6850) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:22:05,230::__init__::232::IOProcess::(_processLogs) (6851) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:05,231::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:05,231::__init__::232::IOProcess::(_processLogs) (6852) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:05,231::__init__::232::IOProcess::(_processLogs) (6852) Queuing response >Thread-14::DEBUG::2014-08-11 15:22:05,232::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:05,232::__init__::232::IOProcess::(_processLogs) (6853) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:05,233::__init__::232::IOProcess::(_processLogs) (6853) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:05,237::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:05,238::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:05,239::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:06,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:06,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131303 s, 78.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:06,871::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:22:06,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`f924ea13-fb2d-4fe0-89d8-98b41b4df4bd`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:06,872::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:22:06,885::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:06,885::task::1191::Storage.TaskManager.Task::(prepare) Task=`f924ea13-fb2d-4fe0-89d8-98b41b4df4bd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:06,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`f924ea13-fb2d-4fe0-89d8-98b41b4df4bd`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:06,886::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:06,886::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:06,886::task::993::Storage.TaskManager.Task::(_decref) Task=`f924ea13-fb2d-4fe0-89d8-98b41b4df4bd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:22:06,897::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:22:06,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`75d67db8-bcca-4a67-9707-8785816c918e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:22:06,898::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:22:06,899::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3bda7365-7475-438f-8360-0bc5cd1a074f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:22:06,899::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:22:06,899::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:22:06,900::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3bda7365-7475-438f-8360-0bc5cd1a074f`::Granted request >Thread-34::DEBUG::2014-08-11 15:22:06,900::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`75d67db8-bcca-4a67-9707-8785816c918e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:22:06,900::task::993::Storage.TaskManager.Task::(_decref) Task=`75d67db8-bcca-4a67-9707-8785816c918e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:22:06,920::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:06,921::task::1191::Storage.TaskManager.Task::(prepare) Task=`75d67db8-bcca-4a67-9707-8785816c918e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:06,921::task::595::Storage.TaskManager.Task::(_updateState) Task=`75d67db8-bcca-4a67-9707-8785816c918e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:22:06,921::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:22:06,921::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:22:06,922::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:22:06,922::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:22:06,922::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:22:06,923::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:22:06,923::task::993::Storage.TaskManager.Task::(_decref) Task=`75d67db8-bcca-4a67-9707-8785816c918e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:08,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:08,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132852 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:10,246::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:10,247::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:10,329::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:10,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:10,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131884 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:12,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:12,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132782 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:15,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:15,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132028 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:15,235::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:15,235::__init__::232::IOProcess::(_processLogs) (6854) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:22:15,236::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:22:15,257::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000500168 s, 910 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:15,261::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:22:15,329::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:15,330::__init__::232::IOProcess::(_processLogs) (6855) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:22:15,331::__init__::232::IOProcess::(_processLogs) (6856) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:15,332::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:15,332::__init__::232::IOProcess::(_processLogs) (6857) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:15,333::__init__::232::IOProcess::(_processLogs) (6857) Queuing response >Thread-14::DEBUG::2014-08-11 15:22:15,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:15,334::__init__::232::IOProcess::(_processLogs) (6858) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:15,337::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:15,338::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:15,340::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:22:16,934::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:22:16,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`513cd169-dc5a-4ea4-8bf3-ebf0be765a4f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:16,936::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:22:16,948::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:16,949::task::1191::Storage.TaskManager.Task::(prepare) Task=`513cd169-dc5a-4ea4-8bf3-ebf0be765a4f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:16,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`513cd169-dc5a-4ea4-8bf3-ebf0be765a4f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:16,949::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:16,949::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:16,950::task::993::Storage.TaskManager.Task::(_decref) Task=`513cd169-dc5a-4ea4-8bf3-ebf0be765a4f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:22:16,959::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:22:16,960::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc1107dc-a7fd-4104-942f-7fb24c1eed36`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:22:16,960::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:22:16,961::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c50a8838-c8ab-4515-8073-336c63978e5f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:22:16,961::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:22:16,962::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:22:16,962::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c50a8838-c8ab-4515-8073-336c63978e5f`::Granted request >Thread-34::DEBUG::2014-08-11 15:22:16,962::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fc1107dc-a7fd-4104-942f-7fb24c1eed36`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:22:16,963::task::993::Storage.TaskManager.Task::(_decref) Task=`fc1107dc-a7fd-4104-942f-7fb24c1eed36`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:22:16,982::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:16,983::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc1107dc-a7fd-4104-942f-7fb24c1eed36`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:16,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc1107dc-a7fd-4104-942f-7fb24c1eed36`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:22:16,983::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:22:16,983::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:22:16,984::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:22:16,984::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:22:16,984::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:22:16,985::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:22:16,985::task::993::Storage.TaskManager.Task::(_decref) Task=`fc1107dc-a7fd-4104-942f-7fb24c1eed36`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:17,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:17,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132525 s, 77.3 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:22:18,187::task::595::Storage.TaskManager.Task::(_updateState) Task=`5572d01c-2551-4ed2-9e7d-8ba9dcf9cce1`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:22:18,188::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:22:18,189::__init__::232::IOProcess::(_processLogs) (6859) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:22:18,191::__init__::232::IOProcess::(_processLogs) (6860) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:22:18,192::__init__::232::IOProcess::(_processLogs) (6860) Queuing response >Thread-55::INFO::2014-08-11 15:22:18,192::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:22:18,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`5572d01c-2551-4ed2-9e7d-8ba9dcf9cce1`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:22:18,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`5572d01c-2551-4ed2-9e7d-8ba9dcf9cce1`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:22:18,193::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:22:18,193::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:22:18,193::task::993::Storage.TaskManager.Task::(_decref) Task=`5572d01c-2551-4ed2-9e7d-8ba9dcf9cce1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:19,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:19,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133443 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:19,863::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb594549-47af-48e4-9890-f059945a7f45`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:19,863::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:22:19,864::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000500168', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:19,864::task::1191::Storage.TaskManager.Task::(prepare) Task=`cb594549-47af-48e4-9890-f059945a7f45`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000500168', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:19,864::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb594549-47af-48e4-9890-f059945a7f45`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:19,864::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:19,864::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:19,865::task::993::Storage.TaskManager.Task::(_decref) Task=`cb594549-47af-48e4-9890-f059945a7f45`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:22:19,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:20,347::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:20,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:20,349::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:21,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:21,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133065 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:23,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:23,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132591 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:25,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:25,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157245 s, 65.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:25,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:22:25,339::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:25,341::__init__::232::IOProcess::(_processLogs) (6861) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:22:25,343::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:25,367::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:25,373::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:22:25,372::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00164284 s, 277 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:25,432::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:25,434::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:22:25,434::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:25,436::__init__::232::IOProcess::(_processLogs) (6862) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:22:25,439::__init__::232::IOProcess::(_processLogs) (6863) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:25,440::__init__::232::IOProcess::(_processLogs) (6864) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:25,441::__init__::232::IOProcess::(_processLogs) (6865) Got request for method 'access' >Thread-115::DEBUG::2014-08-11 15:22:26,999::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:22:27,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`76d5e49a-1072-4cd7-b9cc-0865e112b6e8`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:27,000::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:22:27,013::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:27,014::task::1191::Storage.TaskManager.Task::(prepare) Task=`76d5e49a-1072-4cd7-b9cc-0865e112b6e8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:27,014::task::595::Storage.TaskManager.Task::(_updateState) Task=`76d5e49a-1072-4cd7-b9cc-0865e112b6e8`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:27,014::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:27,015::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:27,015::task::993::Storage.TaskManager.Task::(_decref) Task=`76d5e49a-1072-4cd7-b9cc-0865e112b6e8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:22:27,025::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:22:27,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`afb16528-9dfa-47ea-904a-66b560b97791`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:22:27,026::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:22:27,027::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b2e5db7f-3a4a-4945-a6b2-0562144b6387`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:22:27,027::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:22:27,027::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:22:27,028::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b2e5db7f-3a4a-4945-a6b2-0562144b6387`::Granted request >Thread-34::DEBUG::2014-08-11 15:22:27,028::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`afb16528-9dfa-47ea-904a-66b560b97791`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:22:27,028::task::993::Storage.TaskManager.Task::(_decref) Task=`afb16528-9dfa-47ea-904a-66b560b97791`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:22:27,048::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:27,049::task::1191::Storage.TaskManager.Task::(prepare) Task=`afb16528-9dfa-47ea-904a-66b560b97791`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:27,049::task::595::Storage.TaskManager.Task::(_updateState) Task=`afb16528-9dfa-47ea-904a-66b560b97791`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:22:27,049::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:22:27,049::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:22:27,050::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:22:27,050::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:22:27,050::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:22:27,050::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:22:27,051::task::993::Storage.TaskManager.Task::(_decref) Task=`afb16528-9dfa-47ea-904a-66b560b97791`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:27,379::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:27,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134465 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:29,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:29,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130719 s, 78.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:30,445::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:30,446::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:30,530::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:31,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:31,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0147418 s, 69.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:33,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:33,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131361 s, 78.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:34,981::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f1e7bbe-7ac1-4843-8905-aa88c7f08d1e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:34,981::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:22:34,982::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00164284', 'lastCheck': '9.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:34,982::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f1e7bbe-7ac1-4843-8905-aa88c7f08d1e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00164284', 'lastCheck': '9.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:34,982::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f1e7bbe-7ac1-4843-8905-aa88c7f08d1e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:34,983::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:34,983::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:34,983::task::993::Storage.TaskManager.Task::(_decref) Task=`1f1e7bbe-7ac1-4843-8905-aa88c7f08d1e`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:22:35,005::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:22:35,443::__init__::232::IOProcess::(_processLogs) (6866) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:22:35,445::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:22:35,464::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0004211 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:35,465::__init__::232::IOProcess::(_processLogs) (6867) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:22:35,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:35,531::__init__::232::IOProcess::(_processLogs) (6868) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:35,532::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:35,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:35,533::__init__::232::IOProcess::(_processLogs) (6870) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:35,537::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:35,539::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:35,540::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:35,607::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:35,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0291791 s, 35.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:37,063::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:22:37,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`f54aacf5-ba16-40c2-9a96-efeda59e0077`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:37,064::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:22:37,077::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:37,077::task::1191::Storage.TaskManager.Task::(prepare) Task=`f54aacf5-ba16-40c2-9a96-efeda59e0077`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:37,078::task::595::Storage.TaskManager.Task::(_updateState) Task=`f54aacf5-ba16-40c2-9a96-efeda59e0077`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:37,078::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:37,078::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:37,078::task::993::Storage.TaskManager.Task::(_decref) Task=`f54aacf5-ba16-40c2-9a96-efeda59e0077`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:22:37,089::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:22:37,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`d484c0ca-8380-44e8-b26d-6fbfa2d5006d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:22:37,090::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:22:37,090::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a0df12cf-ba8d-412c-ac37-fbb2b2058a09`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:22:37,091::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:22:37,091::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:22:37,091::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a0df12cf-ba8d-412c-ac37-fbb2b2058a09`::Granted request >Thread-34::DEBUG::2014-08-11 15:22:37,092::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d484c0ca-8380-44e8-b26d-6fbfa2d5006d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:22:37,092::task::993::Storage.TaskManager.Task::(_decref) Task=`d484c0ca-8380-44e8-b26d-6fbfa2d5006d`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:37,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 15:22:38,408::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:38,409::task::1191::Storage.TaskManager.Task::(prepare) Task=`d484c0ca-8380-44e8-b26d-6fbfa2d5006d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:38,410::task::595::Storage.TaskManager.Task::(_updateState) Task=`d484c0ca-8380-44e8-b26d-6fbfa2d5006d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:22:38,411::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:22:38,412::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:22:38,414::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:22:38,415::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:22:38,416::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:22:38,417::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:22:38,417::task::993::Storage.TaskManager.Task::(_decref) Task=`d484c0ca-8380-44e8-b26d-6fbfa2d5006d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:38,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148527 s, 68.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:40,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:40,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135192 s, 75.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:40,545::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:40,546::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:40,547::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:42,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:42,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135067 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:44,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:44,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131998 s, 77.6 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:22:45,534::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:22:45,536::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,537::__init__::232::IOProcess::(_processLogs) (6871) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:22:45,544::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:22:45,545::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,546::__init__::232::IOProcess::(_processLogs) (6872) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 15:22:45,547::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:22:45,548::__init__::232::IOProcess::(_processLogs) (6873) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 15:22:45,549::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:22:45,550::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,550::__init__::232::IOProcess::(_processLogs) (6874) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:45,551::__init__::232::IOProcess::(_processLogs) DEBUG|(6874) Queuing response >Thread-14::DEBUG::2014-08-11 15:22:45,551::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:22:45,552::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:22:45,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,552::__init__::232::IOProcess::(_processLogs) DEBUG|(6875) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:22:45,552::__init__::232::IOProcess::(_processLogs) (6875) Queuing response >Thread-16::DEBUG::2014-08-11 15:22:45,554::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:22:45,555::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:45,556::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::INFO::2014-08-11 15:22:45,557::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:22:45,559::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:22:45,561::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:45,562::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:22:45,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,564::__init__::232::IOProcess::(_processLogs) (6876) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:22:45,566::__init__::232::IOProcess::(_processLogs) DEBUG|(6876) Queuing response >Thread-16::DEBUG::2014-08-11 15:22:45,568::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:45,568::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:22:45,589::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000802459 s, 567 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:45,590::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:22:45,591::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:22:45,591::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,591::__init__::232::IOProcess::(_processLogs) (6877) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:22:45,592::__init__::232::IOProcess::(_processLogs) DEBUG|(6877) Queuing response >Thread-14::DEBUG::2014-08-11 15:22:45,593::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:22:45,594::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:22:45,595::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,595::__init__::232::IOProcess::(_processLogs) (6878) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:45,595::__init__::232::IOProcess::(_processLogs) (6878) Queuing response >Thread-14::DEBUG::2014-08-11 15:22:45,596::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,596::__init__::232::IOProcess::(_processLogs) (6879) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:45,597::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:45,597::__init__::232::IOProcess::(_processLogs) (6880) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:45,597::__init__::232::IOProcess::(_processLogs) (6880) Queuing response >Dummy-33::DEBUG::2014-08-11 15:22:46,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:46,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134264 s, 76.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:48,430::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:22:48,431::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d53a819-f8aa-4989-954a-580c057f7fd7`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:48,431::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:22:48,444::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:48,444::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d53a819-f8aa-4989-954a-580c057f7fd7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:48,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d53a819-f8aa-4989-954a-580c057f7fd7`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:48,444::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:48,445::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:48,445::task::993::Storage.TaskManager.Task::(_decref) Task=`9d53a819-f8aa-4989-954a-580c057f7fd7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:22:48,455::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:22:48,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d850fb2-4cb4-4593-a67a-1c9ece809ecf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:22:48,456::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:22:48,456::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fda75818-0abc-457f-96f7-061a83006567`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:22:48,457::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:22:48,457::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:22:48,457::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fda75818-0abc-457f-96f7-061a83006567`::Granted request >Thread-34::DEBUG::2014-08-11 15:22:48,458::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0d850fb2-4cb4-4593-a67a-1c9ece809ecf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:22:48,458::task::993::Storage.TaskManager.Task::(_decref) Task=`0d850fb2-4cb4-4593-a67a-1c9ece809ecf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:22:48,478::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:48,478::task::1191::Storage.TaskManager.Task::(prepare) Task=`0d850fb2-4cb4-4593-a67a-1c9ece809ecf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:48,478::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d850fb2-4cb4-4593-a67a-1c9ece809ecf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:22:48,479::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:22:48,479::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:22:48,479::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:22:48,479::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:22:48,480::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:22:48,480::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:22:48,480::task::993::Storage.TaskManager.Task::(_decref) Task=`0d850fb2-4cb4-4593-a67a-1c9ece809ecf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:48,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:48,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132347 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:50,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c67e8ab-4238-4077-a5be-5b099712eff1`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:50,100::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:22:50,101::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000802459', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:50,101::task::1191::Storage.TaskManager.Task::(prepare) Task=`4c67e8ab-4238-4077-a5be-5b099712eff1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000802459', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:22:50,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`4c67e8ab-4238-4077-a5be-5b099712eff1`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:50,101::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:50,102::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:50,102::task::993::Storage.TaskManager.Task::(_decref) Task=`4c67e8ab-4238-4077-a5be-5b099712eff1`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:22:50,121::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:50,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:50,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:50,631::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:50,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:50,757::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132371 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:52,780::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:52,813::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131939 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:22:54,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:54,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131606 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:55,599::__init__::232::IOProcess::(_processLogs) (6881) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:22:55,600::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:22:55,620::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418854 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:22:55,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:55,621::__init__::232::IOProcess::(_processLogs) (6882) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:22:55,623::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:55,623::__init__::232::IOProcess::(_processLogs) (6883) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:55,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:55,624::__init__::232::IOProcess::(_processLogs) (6884) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:55,625::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:22:55,625::__init__::232::IOProcess::(_processLogs) (6885) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:22:55,626::__init__::232::IOProcess::(_processLogs) (6885) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:55,638::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:55,639::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:22:55,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:22:56,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:56,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133009 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:22:58,492::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:22:58,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`31aee9db-3b8a-4db1-be61-8ba439d3dfb1`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:22:58,493::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:22:58,506::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:58,506::task::1191::Storage.TaskManager.Task::(prepare) Task=`31aee9db-3b8a-4db1-be61-8ba439d3dfb1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:22:58,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`31aee9db-3b8a-4db1-be61-8ba439d3dfb1`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:22:58,507::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:22:58,507::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:22:58,507::task::993::Storage.TaskManager.Task::(_decref) Task=`31aee9db-3b8a-4db1-be61-8ba439d3dfb1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:22:58,518::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:22:58,519::task::595::Storage.TaskManager.Task::(_updateState) Task=`384fff32-dcf1-499a-a805-5596de8421fb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:22:58,519::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:22:58,520::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`19f51c64-a29c-4b89-acb4-5420d2efbaf0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:22:58,520::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:22:58,521::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:22:58,521::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`19f51c64-a29c-4b89-acb4-5420d2efbaf0`::Granted request >Thread-34::DEBUG::2014-08-11 15:22:58,521::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`384fff32-dcf1-499a-a805-5596de8421fb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:22:58,522::task::993::Storage.TaskManager.Task::(_decref) Task=`384fff32-dcf1-499a-a805-5596de8421fb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:22:58,542::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:58,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`384fff32-dcf1-499a-a805-5596de8421fb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:22:58,543::task::595::Storage.TaskManager.Task::(_updateState) Task=`384fff32-dcf1-499a-a805-5596de8421fb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:22:58,543::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:22:58,543::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:22:58,544::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:22:58,544::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:22:58,544::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:22:58,544::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:22:58,545::task::993::Storage.TaskManager.Task::(_decref) Task=`384fff32-dcf1-499a-a805-5596de8421fb`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:22:58,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:22:58,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133358 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:00,643::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:00,645::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:00,731::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:00,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:01,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134672 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:03,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:03,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013302 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:05,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:05,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130935 s, 78.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:05,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf6faf0e-2b6b-4701-9aa6-b35c26a4e48b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:05,215::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:23:05,215::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418854', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:05,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf6faf0e-2b6b-4701-9aa6-b35c26a4e48b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418854', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:05,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf6faf0e-2b6b-4701-9aa6-b35c26a4e48b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:05,216::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:05,216::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:05,216::task::993::Storage.TaskManager.Task::(_decref) Task=`cf6faf0e-2b6b-4701-9aa6-b35c26a4e48b`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:23:05,236::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:05,627::__init__::232::IOProcess::(_processLogs) (6886) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:23:05,628::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:23:05,648::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418303 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:05,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:05,731::__init__::232::IOProcess::(_processLogs) (6887) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:05,733::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:05,733::__init__::232::IOProcess::(_processLogs) (6888) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:05,734::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:05,734::__init__::232::IOProcess::(_processLogs) (6889) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:05,735::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6889) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:05,735::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:05,736::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:05,736::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:05,736::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6890) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:05,738::__init__::232::IOProcess::(_processLogs) (6890) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:05,739::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:05,742::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:05,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:07,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:07,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013235 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:08,556::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:23:08,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f146666-384d-49c7-ba96-0a7fc2321688`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:08,557::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:23:08,570::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:08,571::task::1191::Storage.TaskManager.Task::(prepare) Task=`6f146666-384d-49c7-ba96-0a7fc2321688`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:08,571::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f146666-384d-49c7-ba96-0a7fc2321688`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:08,571::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:08,571::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:08,571::task::993::Storage.TaskManager.Task::(_decref) Task=`6f146666-384d-49c7-ba96-0a7fc2321688`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:23:08,581::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:23:08,582::task::595::Storage.TaskManager.Task::(_updateState) Task=`d79bc665-d297-43d9-9f0d-92687b1eee12`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:23:08,582::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:23:08,583::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`99c5d896-2cbb-431c-815e-8e4beaffd5b9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:23:08,583::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:23:08,583::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:23:08,584::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`99c5d896-2cbb-431c-815e-8e4beaffd5b9`::Granted request >Thread-34::DEBUG::2014-08-11 15:23:08,584::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d79bc665-d297-43d9-9f0d-92687b1eee12`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:23:08,584::task::993::Storage.TaskManager.Task::(_decref) Task=`d79bc665-d297-43d9-9f0d-92687b1eee12`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:23:08,604::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:08,605::task::1191::Storage.TaskManager.Task::(prepare) Task=`d79bc665-d297-43d9-9f0d-92687b1eee12`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:08,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`d79bc665-d297-43d9-9f0d-92687b1eee12`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:23:08,605::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:23:08,605::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:23:08,606::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:23:08,606::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:23:08,607::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:23:08,607::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:23:08,608::task::993::Storage.TaskManager.Task::(_decref) Task=`d79bc665-d297-43d9-9f0d-92687b1eee12`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:23:09,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:09,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013251 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:10,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:10,752::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:10,832::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:11,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:11,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133301 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:13,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:13,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132306 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:15,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:15,424::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131754 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:15,742::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:15,743::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:15,743::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:15,743::__init__::232::IOProcess::(_processLogs) (6891) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:15,744::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6891) Queuing response >Thread-16::DEBUG::2014-08-11 15:23:15,744::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:23:15,764::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418937 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:15,765::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:15,831::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:15,832::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:15,834::__init__::232::IOProcess::(_processLogs) (6892) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:15,834::__init__::232::IOProcess::(_processLogs) (6892) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:15,834::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:15,835::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:15,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:15,835::__init__::232::IOProcess::(_processLogs) (6893) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:15,835::__init__::232::IOProcess::(_processLogs) (6893) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:15,836::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:15,837::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:15,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:15,838::__init__::232::IOProcess::(_processLogs) (6894) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:15,838::__init__::232::IOProcess::(_processLogs) (6894) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:15,838::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:15,839::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:15,840::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:15,842::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:15,844::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:15,843::__init__::232::IOProcess::(_processLogs) (6895) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:15,846::__init__::232::IOProcess::(_processLogs) (6895) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:15,847::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:17,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:17,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132151 s, 77.5 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:23:18,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`db84f7ea-a1fc-4d04-8334-a61a694ecc04`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:23:18,300::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:23:18,301::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:18,301::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:18,302::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:18,302::__init__::232::IOProcess::(_processLogs) (6896) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:23:18,303::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6896) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:18,304::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:18,304::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:18,304::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:18,305::__init__::232::IOProcess::(_processLogs) (6897) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:23:18,305::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:23:18,305::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6897) Queuing response >Thread-55::DEBUG::2014-08-11 15:23:18,307::task::1191::Storage.TaskManager.Task::(prepare) Task=`db84f7ea-a1fc-4d04-8334-a61a694ecc04`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:23:18,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`db84f7ea-a1fc-4d04-8334-a61a694ecc04`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:23:18,307::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:23:18,307::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:23:18,308::task::993::Storage.TaskManager.Task::(_decref) Task=`db84f7ea-a1fc-4d04-8334-a61a694ecc04`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:23:18,620::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:23:18,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`89c90757-ce10-4227-9d1c-73a36b456488`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:18,621::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:23:18,634::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:18,634::task::1191::Storage.TaskManager.Task::(prepare) Task=`89c90757-ce10-4227-9d1c-73a36b456488`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:18,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`89c90757-ce10-4227-9d1c-73a36b456488`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:18,634::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:18,635::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:18,635::task::993::Storage.TaskManager.Task::(_decref) Task=`89c90757-ce10-4227-9d1c-73a36b456488`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:23:18,645::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:23:18,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`16d1a582-ad55-4e83-b6d3-b660b2916c41`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:23:18,646::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:23:18,647::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9c210cb4-824e-465c-b1c7-3c4cfcb2fd54`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:23:18,647::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:23:18,648::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:23:18,648::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9c210cb4-824e-465c-b1c7-3c4cfcb2fd54`::Granted request >Thread-34::DEBUG::2014-08-11 15:23:18,648::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`16d1a582-ad55-4e83-b6d3-b660b2916c41`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:23:18,649::task::993::Storage.TaskManager.Task::(_decref) Task=`16d1a582-ad55-4e83-b6d3-b660b2916c41`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:23:18,668::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:18,669::task::1191::Storage.TaskManager.Task::(prepare) Task=`16d1a582-ad55-4e83-b6d3-b660b2916c41`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:18,669::task::595::Storage.TaskManager.Task::(_updateState) Task=`16d1a582-ad55-4e83-b6d3-b660b2916c41`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:23:18,669::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:23:18,669::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:23:18,670::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:23:18,670::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:23:18,670::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:23:18,671::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:23:18,671::task::993::Storage.TaskManager.Task::(_decref) Task=`16d1a582-ad55-4e83-b6d3-b660b2916c41`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:23:19,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:19,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132919 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:20,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`55e7fd1f-53c9-4747-a78b-055f0f8f33e3`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:20,358::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:23:20,358::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418937', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:20,359::task::1191::Storage.TaskManager.Task::(prepare) Task=`55e7fd1f-53c9-4747-a78b-055f0f8f33e3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418937', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:20,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`55e7fd1f-53c9-4747-a78b-055f0f8f33e3`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:20,359::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:20,359::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:20,360::task::993::Storage.TaskManager.Task::(_decref) Task=`55e7fd1f-53c9-4747-a78b-055f0f8f33e3`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:23:20,379::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:20,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:20,856::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:20,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:21,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:21,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132999 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:23,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:23,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132584 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:25,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:25,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0244563 s, 41.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:25,844::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:25,844::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:25,845::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:25,845::__init__::232::IOProcess::(_processLogs) (6898) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:25,846::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6898) Queuing response >Thread-16::DEBUG::2014-08-11 15:23:25,847::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:25,866::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:25,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:23:25,868::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000406296 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:25,934::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:25,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:25,937::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:25,938::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:25,939::__init__::232::IOProcess::(_processLogs) (6899) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:25,941::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6899) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:25,942::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:25,942::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:25,943::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:25,943::__init__::232::IOProcess::(_processLogs) (6900) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:25,943::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6900) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:25,944::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:25,945::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:25,945::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:25,945::__init__::232::IOProcess::(_processLogs) (6901) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:25,945::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6901) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:25,946::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:25,946::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:25,947::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:25,947::__init__::232::IOProcess::(_processLogs) (6902) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:25,947::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6902) Queuing response >Dummy-33::DEBUG::2014-08-11 15:23:27,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:27,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134186 s, 76.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:28,686::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:23:28,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`b41394b3-0049-4844-9bf9-a8cca5d1873b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:28,688::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:23:28,701::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:28,701::task::1191::Storage.TaskManager.Task::(prepare) Task=`b41394b3-0049-4844-9bf9-a8cca5d1873b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:28,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`b41394b3-0049-4844-9bf9-a8cca5d1873b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:28,702::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:28,702::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:28,702::task::993::Storage.TaskManager.Task::(_decref) Task=`b41394b3-0049-4844-9bf9-a8cca5d1873b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:23:28,714::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:23:28,714::task::595::Storage.TaskManager.Task::(_updateState) Task=`2dca2572-5378-4779-993a-928ee4cf7060`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:23:28,715::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:23:28,715::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a1e101b3-96c0-44db-a370-9deb3e36897d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:23:28,716::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:23:28,716::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:23:28,716::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a1e101b3-96c0-44db-a370-9deb3e36897d`::Granted request >Thread-34::DEBUG::2014-08-11 15:23:28,717::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2dca2572-5378-4779-993a-928ee4cf7060`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:23:28,717::task::993::Storage.TaskManager.Task::(_decref) Task=`2dca2572-5378-4779-993a-928ee4cf7060`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:23:28,737::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:28,737::task::1191::Storage.TaskManager.Task::(prepare) Task=`2dca2572-5378-4779-993a-928ee4cf7060`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:28,738::task::595::Storage.TaskManager.Task::(_updateState) Task=`2dca2572-5378-4779-993a-928ee4cf7060`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:23:28,738::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:23:28,738::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:23:28,738::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:23:28,739::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:23:28,739::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:23:28,739::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:23:28,739::task::993::Storage.TaskManager.Task::(_decref) Task=`2dca2572-5378-4779-993a-928ee4cf7060`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:23:29,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:29,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132945 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:30,942::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:30,943::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:31,033::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:31,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:31,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133497 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:33,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:33,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132353 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:35,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`4049770f-6d6a-4113-b41d-bb3e205e101f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:35,501::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:23:35,502::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406296', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:35,502::task::1191::Storage.TaskManager.Task::(prepare) Task=`4049770f-6d6a-4113-b41d-bb3e205e101f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406296', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:35,502::task::595::Storage.TaskManager.Task::(_updateState) Task=`4049770f-6d6a-4113-b41d-bb3e205e101f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:35,502::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:35,503::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:35,503::task::993::Storage.TaskManager.Task::(_decref) Task=`4049770f-6d6a-4113-b41d-bb3e205e101f`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:23:35,523::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:35,949::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:35,950::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 15:23:35,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:23:36,034::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:23:36,053::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:23:36,055::__init__::232::IOProcess::(_processLogs) (6903) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:36,077::__init__::232::IOProcess::(_processLogs) (6903) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:36,079::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:23:36,081::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00134994 s, 337 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:36,085::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:36,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:36,088::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:36,091::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:36,093::__init__::232::IOProcess::(_processLogs) (6904) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:36,095::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6904) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:36,100::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:36,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:36,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:36,105::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:36,108::__init__::232::IOProcess::(_processLogs) (6905) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:36,112::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6905) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:36,116::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:36,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 15:23:36,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0599588 s, 17.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:36,120::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:36,122::__init__::232::IOProcess::(_processLogs) (6906) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:36,124::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6906) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:36,126::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:36,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:36,130::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:36,131::__init__::232::IOProcess::(_processLogs) (6907) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:36,134::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6907) Queuing response >Dummy-33::DEBUG::2014-08-11 15:23:38,338::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:38,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133568 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:38,750::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:23:38,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebadbb53-c9f7-4cd3-9f7a-1c77eff32600`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:38,752::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:23:38,764::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:38,765::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebadbb53-c9f7-4cd3-9f7a-1c77eff32600`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:38,765::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebadbb53-c9f7-4cd3-9f7a-1c77eff32600`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:38,765::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:38,765::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:38,766::task::993::Storage.TaskManager.Task::(_decref) Task=`ebadbb53-c9f7-4cd3-9f7a-1c77eff32600`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:23:38,776::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:23:38,777::task::595::Storage.TaskManager.Task::(_updateState) Task=`790b03e4-9510-446a-a56d-0f24c14a1776`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:23:38,777::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:23:38,778::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7a64e2c8-e990-4e75-bac3-f66bb3fae816`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:23:38,778::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:23:38,778::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:23:38,779::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7a64e2c8-e990-4e75-bac3-f66bb3fae816`::Granted request >Thread-34::DEBUG::2014-08-11 15:23:38,779::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`790b03e4-9510-446a-a56d-0f24c14a1776`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:23:38,779::task::993::Storage.TaskManager.Task::(_decref) Task=`790b03e4-9510-446a-a56d-0f24c14a1776`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:23:38,799::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:38,800::task::1191::Storage.TaskManager.Task::(prepare) Task=`790b03e4-9510-446a-a56d-0f24c14a1776`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:38,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`790b03e4-9510-446a-a56d-0f24c14a1776`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:23:38,800::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:23:38,800::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:23:38,801::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:23:38,801::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:23:38,801::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:23:38,801::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:23:38,802::task::993::Storage.TaskManager.Task::(_decref) Task=`790b03e4-9510-446a-a56d-0f24c14a1776`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:23:40,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:40,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132577 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:41,114::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:41,116::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:41,133::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:42,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:42,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132539 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:44,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:44,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132246 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:46,138::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:46,142::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:46,142::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:23:46,143::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:23:46,234::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:46,254::__init__::232::IOProcess::(_processLogs) (6908) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:46,256::__init__::232::IOProcess::(_processLogs) (6908) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:46,258::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:23:46,258::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000528218 s, 861 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:46,261::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:46,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:46,263::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:46,265::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6909) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:46,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:23:46,267::__init__::232::IOProcess::(_processLogs) (6909) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:46,269::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:46,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:46,271::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:46,271::__init__::232::IOProcess::(_processLogs) (6910) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:46,271::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6910) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:46,272::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:46,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:46,273::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:46,274::__init__::232::IOProcess::(_processLogs) (6911) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:46,274::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6911) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:46,275::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:46,276::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:46,278::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:46,278::__init__::232::IOProcess::(_processLogs) (6912) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:46,279::__init__::232::IOProcess::(_processLogs) (6912) Queuing response >Dummy-33::DEBUG::2014-08-11 15:23:46,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:46,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132507 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:48,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:48,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132918 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:48,815::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:23:48,816::task::595::Storage.TaskManager.Task::(_updateState) Task=`acc3756f-0cb2-4b20-86c0-dc893505ebfc`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:48,816::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:23:48,829::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:48,829::task::1191::Storage.TaskManager.Task::(prepare) Task=`acc3756f-0cb2-4b20-86c0-dc893505ebfc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:48,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`acc3756f-0cb2-4b20-86c0-dc893505ebfc`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:48,830::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:48,830::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:48,830::task::993::Storage.TaskManager.Task::(_decref) Task=`acc3756f-0cb2-4b20-86c0-dc893505ebfc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:23:48,842::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:23:48,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`252e4d27-860f-4ae1-93c2-245631511b75`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:23:48,843::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:23:48,843::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`228b36f1-f8c1-4aa0-96b8-158c47a5e53a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:23:48,844::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:23:48,844::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:23:48,844::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`228b36f1-f8c1-4aa0-96b8-158c47a5e53a`::Granted request >Thread-34::DEBUG::2014-08-11 15:23:48,845::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`252e4d27-860f-4ae1-93c2-245631511b75`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:23:48,845::task::993::Storage.TaskManager.Task::(_decref) Task=`252e4d27-860f-4ae1-93c2-245631511b75`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:23:48,866::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:48,867::task::1191::Storage.TaskManager.Task::(prepare) Task=`252e4d27-860f-4ae1-93c2-245631511b75`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:48,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`252e4d27-860f-4ae1-93c2-245631511b75`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:23:48,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:23:48,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:23:48,869::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:23:48,870::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:23:48,870::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:23:48,870::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:23:48,871::task::993::Storage.TaskManager.Task::(_decref) Task=`252e4d27-860f-4ae1-93c2-245631511b75`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:23:50,634::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4707695-139b-4e1e-84fe-7452b3579eb5`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:50,634::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:23:50,635::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000528218', 'lastCheck': '4.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:50,635::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4707695-139b-4e1e-84fe-7452b3579eb5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000528218', 'lastCheck': '4.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:23:50,635::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4707695-139b-4e1e-84fe-7452b3579eb5`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:50,635::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:50,636::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:50,636::task::993::Storage.TaskManager.Task::(_decref) Task=`f4707695-139b-4e1e-84fe-7452b3579eb5`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:23:50,655::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:50,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:50,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133088 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:51,277::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:51,278::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:51,333::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:23:52,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:52,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131948 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:23:54,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:54,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132539 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:56,279::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:56,279::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:56,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:56,332::__init__::232::IOProcess::(_processLogs) (6913) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:56,333::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6913) Queuing response >Thread-16::DEBUG::2014-08-11 15:23:56,334::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:56,354::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:56,356::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:23:56,358::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:23:56,358::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000412731 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:23:56,362::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:56,362::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:56,362::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:56,363::__init__::232::IOProcess::(_processLogs) (6914) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:23:56,363::__init__::232::IOProcess::(_processLogs) (6914) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:56,364::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:56,365::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:56,365::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:56,366::__init__::232::IOProcess::(_processLogs) (6915) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:56,366::__init__::232::IOProcess::(_processLogs) (6915) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:56,366::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:56,367::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:56,367::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:56,368::__init__::232::IOProcess::(_processLogs) (6916) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:56,368::__init__::232::IOProcess::(_processLogs) (6916) Queuing response >Thread-14::DEBUG::2014-08-11 15:23:56,368::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:23:56,369::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:23:56,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:23:56,370::__init__::232::IOProcess::(_processLogs) (6917) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:23:56,371::__init__::232::IOProcess::(_processLogs) (6917) Queuing response >Dummy-33::DEBUG::2014-08-11 15:23:56,842::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:23:56,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139871 s, 73.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:23:58,885::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-115::DEBUG::2014-08-11 15:23:58,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d6aaf0a-5ce0-49bf-af91-87f788d0b679`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:23:58,886::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:23:58,899::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:58,899::task::1191::Storage.TaskManager.Task::(prepare) Task=`5d6aaf0a-5ce0-49bf-af91-87f788d0b679`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:23:58,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d6aaf0a-5ce0-49bf-af91-87f788d0b679`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:23:58,900::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:23:58,900::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:23:58,900::task::993::Storage.TaskManager.Task::(_decref) Task=`5d6aaf0a-5ce0-49bf-af91-87f788d0b679`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:23:58,903::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::DEBUG::2014-08-11 15:23:58,923::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 15:23:58,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0784166-c0ab-4524-8520-fb9322968a47`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:23:58,924::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:23:58,925::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`69ced345-693b-45c8-8a52-3ada55eec4b8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:23:58,925::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:23:58,926::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:23:58,926::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`69ced345-693b-45c8-8a52-3ada55eec4b8`::Granted request >Thread-34::DEBUG::2014-08-11 15:23:58,927::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d0784166-c0ab-4524-8520-fb9322968a47`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:23:58,927::task::993::Storage.TaskManager.Task::(_decref) Task=`d0784166-c0ab-4524-8520-fb9322968a47`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 15:23:58,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.014039 s, 72.9 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 15:23:58,954::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:58,956::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0784166-c0ab-4524-8520-fb9322968a47`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:23:58,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0784166-c0ab-4524-8520-fb9322968a47`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:23:58,957::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:23:58,958::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:23:58,960::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:23:58,962::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:23:58,963::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:23:58,964::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:23:58,966::task::993::Storage.TaskManager.Task::(_decref) Task=`d0784166-c0ab-4524-8520-fb9322968a47`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:24:00,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:01,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133235 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:01,364::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:01,365::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:01,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:03,037::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:03,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133322 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:05,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:05,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132262 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:05,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8d22228-0477-404a-9be2-eec8c00490d5`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:05,751::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:24:05,752::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412731', 'lastCheck': '9.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:05,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8d22228-0477-404a-9be2-eec8c00490d5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000412731', 'lastCheck': '9.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:05,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8d22228-0477-404a-9be2-eec8c00490d5`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:05,753::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:05,753::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:05,753::task::993::Storage.TaskManager.Task::(_decref) Task=`c8d22228-0477-404a-9be2-eec8c00490d5`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:24:05,784::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:24:06,371::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:06,372::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:24:06,373::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:24:06,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:06,452::__init__::232::IOProcess::(_processLogs) (6918) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:06,454::__init__::232::IOProcess::(_processLogs) (6918) Queuing response >Thread-16::DEBUG::2014-08-11 15:24:06,454::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000571893 s, 796 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:06,456::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:24:06,458::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:06,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:06,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:06,462::__init__::232::IOProcess::(_processLogs) (6919) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:06,463::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:24:06,465::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6919) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:06,467::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:06,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:06,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:06,470::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:24:06,471::__init__::232::IOProcess::(_processLogs) (6920) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:06,473::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6920) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:06,475::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:06,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:06,475::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:06,476::__init__::232::IOProcess::(_processLogs) (6921) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:06,476::__init__::232::IOProcess::(_processLogs) (6921) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:06,477::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:06,477::__init__::232::IOProcess::(_processLogs) (6922) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:24:07,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:07,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133064 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:08,985::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:24:08,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`564826c5-a33d-4c64-874f-ae8a31074a6a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:08,986::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:24:08,999::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:08,999::task::1191::Storage.TaskManager.Task::(prepare) Task=`564826c5-a33d-4c64-874f-ae8a31074a6a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:08,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`564826c5-a33d-4c64-874f-ae8a31074a6a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:09,000::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:09,000::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:09,000::task::993::Storage.TaskManager.Task::(_decref) Task=`564826c5-a33d-4c64-874f-ae8a31074a6a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:24:09,012::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:24:09,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6d08ddb-0069-40e7-8316-9c206364b509`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:24:09,013::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:24:09,014::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8c1d8b4d-074e-4813-8663-be1b2ba95199`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:24:09,014::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:24:09,015::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:24:09,015::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8c1d8b4d-074e-4813-8663-be1b2ba95199`::Granted request >Thread-34::DEBUG::2014-08-11 15:24:09,017::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6d08ddb-0069-40e7-8316-9c206364b509`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:24:09,018::task::993::Storage.TaskManager.Task::(_decref) Task=`b6d08ddb-0069-40e7-8316-9c206364b509`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:24:09,038::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:09,038::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6d08ddb-0069-40e7-8316-9c206364b509`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:09,038::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6d08ddb-0069-40e7-8316-9c206364b509`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:24:09,038::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:24:09,039::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:24:09,039::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:24:09,039::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:24:09,040::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:24:09,040::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:24:09,040::task::993::Storage.TaskManager.Task::(_decref) Task=`b6d08ddb-0069-40e7-8316-9c206364b509`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:24:09,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:09,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131546 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:11,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:11,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131771 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:11,481::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:11,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:11,533::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:13,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:13,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132265 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:15,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:15,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132521 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:16,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:16,480::__init__::232::IOProcess::(_processLogs) (6923) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:24:16,481::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:24:16,499::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415356 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:16,501::__init__::232::IOProcess::(_processLogs) (6924) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:16,534::__init__::232::IOProcess::(_processLogs) (6925) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:16,535::__init__::232::IOProcess::(_processLogs) (6926) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:16,536::__init__::232::IOProcess::(_processLogs) (6927) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:16,540::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:16,541::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:16,542::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:17,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:17,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133307 s, 76.8 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:24:18,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`732d26b2-9f9c-46c8-9327-bc3f78d002f5`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:24:18,423::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:24:18,424::__init__::232::IOProcess::(_processLogs) (6928) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:24:18,427::__init__::232::IOProcess::(_processLogs) (6929) Got request for method 'stat' >Thread-55::INFO::2014-08-11 15:24:18,427::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:24:18,428::task::1191::Storage.TaskManager.Task::(prepare) Task=`732d26b2-9f9c-46c8-9327-bc3f78d002f5`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-55::DEBUG::2014-08-11 15:24:18,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`732d26b2-9f9c-46c8-9327-bc3f78d002f5`::moving from state preparing -> state finished >Thread-55::DEBUG::2014-08-11 15:24:18,428::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:24:18,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:24:18,428::task::993::Storage.TaskManager.Task::(_decref) Task=`732d26b2-9f9c-46c8-9327-bc3f78d002f5`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:24:19,052::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:24:19,053::task::595::Storage.TaskManager.Task::(_updateState) Task=`b46b6d39-1293-498c-aa41-096c292b1bbf`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:19,053::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:24:19,066::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:19,066::task::1191::Storage.TaskManager.Task::(prepare) Task=`b46b6d39-1293-498c-aa41-096c292b1bbf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:19,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`b46b6d39-1293-498c-aa41-096c292b1bbf`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:19,067::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:19,067::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:19,067::task::993::Storage.TaskManager.Task::(_decref) Task=`b46b6d39-1293-498c-aa41-096c292b1bbf`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:24:19,077::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:24:19,078::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8f198b4-b2a0-4044-8487-95fd6a144fb1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:24:19,078::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:24:19,079::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`60303352-4a67-4644-ac98-076d80223def`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:24:19,079::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:24:19,079::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:24:19,080::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`60303352-4a67-4644-ac98-076d80223def`::Granted request >Thread-34::DEBUG::2014-08-11 15:24:19,080::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c8f198b4-b2a0-4044-8487-95fd6a144fb1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:24:19,080::task::993::Storage.TaskManager.Task::(_decref) Task=`c8f198b4-b2a0-4044-8487-95fd6a144fb1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:24:19,100::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:19,101::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8f198b4-b2a0-4044-8487-95fd6a144fb1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:19,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8f198b4-b2a0-4044-8487-95fd6a144fb1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:24:19,101::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:24:19,101::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:24:19,102::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:24:19,102::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:24:19,102::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:24:19,102::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:24:19,103::task::993::Storage.TaskManager.Task::(_decref) Task=`c8f198b4-b2a0-4044-8487-95fd6a144fb1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:24:19,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:19,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133038 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:20,873::task::595::Storage.TaskManager.Task::(_updateState) Task=`207193a7-41a7-4910-9abe-7c34b0ee804d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:20,873::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:24:20,873::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415356', 'lastCheck': '4.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:20,874::task::1191::Storage.TaskManager.Task::(prepare) Task=`207193a7-41a7-4910-9abe-7c34b0ee804d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415356', 'lastCheck': '4.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:20,874::task::595::Storage.TaskManager.Task::(_updateState) Task=`207193a7-41a7-4910-9abe-7c34b0ee804d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:20,874::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:20,874::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:20,874::task::993::Storage.TaskManager.Task::(_decref) Task=`207193a7-41a7-4910-9abe-7c34b0ee804d`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:24:20,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:21,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:21,553::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:21,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133024 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:21,637::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:21,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:23,665::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:23,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132283 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:25,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:25,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131465 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:26,538::__init__::232::IOProcess::(_processLogs) (6930) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:24:26,539::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:24:26,557::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422412 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:26,558::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:26,634::__init__::232::IOProcess::(_processLogs) (6931) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:26,635::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:26,635::__init__::232::IOProcess::(_processLogs) (6932) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:26,636::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:26,637::__init__::232::IOProcess::(_processLogs) (6933) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:26,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:26,638::__init__::232::IOProcess::(_processLogs) (6934) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:26,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:26,651::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:26,652::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:27,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:27,809::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013279 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:29,115::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:24:29,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`2260acbe-4a04-4891-a21c-4d38621310f0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:29,117::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:24:29,130::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:29,131::task::1191::Storage.TaskManager.Task::(prepare) Task=`2260acbe-4a04-4891-a21c-4d38621310f0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:29,131::task::595::Storage.TaskManager.Task::(_updateState) Task=`2260acbe-4a04-4891-a21c-4d38621310f0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:29,131::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:29,131::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:29,131::task::993::Storage.TaskManager.Task::(_decref) Task=`2260acbe-4a04-4891-a21c-4d38621310f0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:24:29,143::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:24:29,144::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e9973cb-8a45-4951-84d5-133ed6356718`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:24:29,144::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:24:29,145::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`590f9a73-33ea-4f39-a3ee-cfcd9842aae3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:24:29,145::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:24:29,146::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:24:29,146::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`590f9a73-33ea-4f39-a3ee-cfcd9842aae3`::Granted request >Thread-34::DEBUG::2014-08-11 15:24:29,147::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3e9973cb-8a45-4951-84d5-133ed6356718`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:24:29,147::task::993::Storage.TaskManager.Task::(_decref) Task=`3e9973cb-8a45-4951-84d5-133ed6356718`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:24:29,167::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:29,167::task::1191::Storage.TaskManager.Task::(prepare) Task=`3e9973cb-8a45-4951-84d5-133ed6356718`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:29,167::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e9973cb-8a45-4951-84d5-133ed6356718`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:24:29,168::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:24:29,168::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:24:29,168::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:24:29,168::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:24:29,169::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:24:29,169::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:24:29,169::task::993::Storage.TaskManager.Task::(_decref) Task=`3e9973cb-8a45-4951-84d5-133ed6356718`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:24:29,831::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:29,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131815 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:31,659::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:31,660::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:31,674::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:31,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:31,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133485 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:34,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:34,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132233 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:35,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fa6ec58-759c-474f-80de-cbe2e84b2ccc`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:35,986::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:24:35,986::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422412', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:35,987::task::1191::Storage.TaskManager.Task::(prepare) Task=`8fa6ec58-759c-474f-80de-cbe2e84b2ccc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000422412', 'lastCheck': '9.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:35,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fa6ec58-759c-474f-80de-cbe2e84b2ccc`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:35,987::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:35,987::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:35,987::task::993::Storage.TaskManager.Task::(_decref) Task=`8fa6ec58-759c-474f-80de-cbe2e84b2ccc`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:24:36,008::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:36,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:36,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013176 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:36,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:36,640::__init__::232::IOProcess::(_processLogs) (6935) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:36,641::__init__::232::IOProcess::(_processLogs) (6935) Queuing response >Thread-16::DEBUG::2014-08-11 15:24:36,641::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:24:36,660::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042233 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:36,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:36,662::__init__::232::IOProcess::(_processLogs) (6936) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:36,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:36,663::__init__::232::IOProcess::(_processLogs) (6937) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:36,664::__init__::232::IOProcess::(_processLogs) DEBUG|(6937) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:36,664::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:36,665::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:36,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:36,665::__init__::232::IOProcess::(_processLogs) DEBUG|(6938) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:36,666::__init__::232::IOProcess::(_processLogs) (6938) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:36,666::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:36,667::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:36,667::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:36,667::__init__::232::IOProcess::(_processLogs) DEBUG|(6939) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:36,667::__init__::232::IOProcess::(_processLogs) (6939) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:36,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:36,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:36,755::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:38,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:38,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134167 s, 76.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:39,182::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:24:39,183::task::595::Storage.TaskManager.Task::(_updateState) Task=`91240339-2ccd-48c5-bf97-0e6d645c05d6`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:39,183::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:24:39,196::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:39,196::task::1191::Storage.TaskManager.Task::(prepare) Task=`91240339-2ccd-48c5-bf97-0e6d645c05d6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:39,196::task::595::Storage.TaskManager.Task::(_updateState) Task=`91240339-2ccd-48c5-bf97-0e6d645c05d6`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:39,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:39,197::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:39,197::task::993::Storage.TaskManager.Task::(_decref) Task=`91240339-2ccd-48c5-bf97-0e6d645c05d6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:24:39,207::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:24:39,208::task::595::Storage.TaskManager.Task::(_updateState) Task=`b45715c4-f69f-4dd6-b162-8002ee6db666`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:24:39,208::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:24:39,209::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`246c8e14-2733-46e0-81ea-6a0e50f46257`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:24:39,209::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:24:39,209::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:24:39,210::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`246c8e14-2733-46e0-81ea-6a0e50f46257`::Granted request >Thread-34::DEBUG::2014-08-11 15:24:39,210::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b45715c4-f69f-4dd6-b162-8002ee6db666`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:24:39,210::task::993::Storage.TaskManager.Task::(_decref) Task=`b45715c4-f69f-4dd6-b162-8002ee6db666`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:24:39,230::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:39,231::task::1191::Storage.TaskManager.Task::(prepare) Task=`b45715c4-f69f-4dd6-b162-8002ee6db666`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:39,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`b45715c4-f69f-4dd6-b162-8002ee6db666`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:24:39,231::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:24:39,231::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:24:39,232::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:24:39,232::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:24:39,232::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:24:39,232::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:24:39,233::task::993::Storage.TaskManager.Task::(_decref) Task=`b45715c4-f69f-4dd6-b162-8002ee6db666`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:24:40,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:40,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132075 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:41,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:41,762::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:41,763::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:42,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:42,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134556 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:44,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:44,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013207 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:46,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:46,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132427 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:46,669::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:46,670::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:46,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:46,670::__init__::232::IOProcess::(_processLogs) (6940) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:46,671::__init__::232::IOProcess::(_processLogs) (6940) Queuing response >Thread-16::DEBUG::2014-08-11 15:24:46,671::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:24:46,691::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000430895 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:46,692::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:46,692::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:46,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:46,693::__init__::232::IOProcess::(_processLogs) (6941) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:46,693::__init__::232::IOProcess::(_processLogs) (6941) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:46,694::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:46,695::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:46,695::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:46,695::__init__::232::IOProcess::(_processLogs) DEBUG|(6942) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:46,695::__init__::232::IOProcess::(_processLogs) (6942) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:46,696::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:46,696::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:46,697::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:46,697::__init__::232::IOProcess::(_processLogs) DEBUG|(6943) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:46,697::__init__::232::IOProcess::(_processLogs) (6943) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:46,698::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:46,698::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:46,699::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:46,699::__init__::232::IOProcess::(_processLogs) DEBUG|(6944) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:46,699::__init__::232::IOProcess::(_processLogs) (6944) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:46,770::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:46,771::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:46,772::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:48,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:48,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132944 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:49,244::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-115::DEBUG::2014-08-11 15:24:49,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`09fe721f-df94-4e5e-9b0c-e8ba38d02684`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:49,246::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:24:49,258::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:49,259::task::1191::Storage.TaskManager.Task::(prepare) Task=`09fe721f-df94-4e5e-9b0c-e8ba38d02684`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:49,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`09fe721f-df94-4e5e-9b0c-e8ba38d02684`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:49,259::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:49,259::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:49,260::task::993::Storage.TaskManager.Task::(_decref) Task=`09fe721f-df94-4e5e-9b0c-e8ba38d02684`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:24:49,270::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5d601cfe] >Thread-34::DEBUG::2014-08-11 15:24:49,271::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7cf2453-6015-4605-b7e6-e6d8b94cd5a0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:24:49,271::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:24:49,272::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c2facb63-c242-4426-9402-b01698f5724f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:24:49,272::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:24:49,273::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:24:49,273::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c2facb63-c242-4426-9402-b01698f5724f`::Granted request >Thread-34::DEBUG::2014-08-11 15:24:49,273::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e7cf2453-6015-4605-b7e6-e6d8b94cd5a0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:24:49,274::task::993::Storage.TaskManager.Task::(_decref) Task=`e7cf2453-6015-4605-b7e6-e6d8b94cd5a0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:24:49,297::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:49,297::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7cf2453-6015-4605-b7e6-e6d8b94cd5a0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:49,297::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7cf2453-6015-4605-b7e6-e6d8b94cd5a0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:24:49,297::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:24:49,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:24:49,298::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:24:49,298::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:24:49,299::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:24:49,299::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:24:49,299::task::993::Storage.TaskManager.Task::(_decref) Task=`e7cf2453-6015-4605-b7e6-e6d8b94cd5a0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:24:50,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:50,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132246 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:51,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cc40288-c5dc-4c7c-ab91-45b8b85db5c8`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:51,101::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:24:51,101::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000430895', 'lastCheck': '4.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:51,101::task::1191::Storage.TaskManager.Task::(prepare) Task=`2cc40288-c5dc-4c7c-ab91-45b8b85db5c8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000430895', 'lastCheck': '4.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:24:51,102::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cc40288-c5dc-4c7c-ab91-45b8b85db5c8`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:51,102::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:51,102::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:51,102::task::993::Storage.TaskManager.Task::(_decref) Task=`2cc40288-c5dc-4c7c-ab91-45b8b85db5c8`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:24:51,122::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:51,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:51,776::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:51,777::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:52,547::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:52,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133335 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:54,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:54,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131587 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:24:56,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:56,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132274 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:56,702::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:56,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:56,704::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:56,706::__init__::232::IOProcess::(_processLogs) (6945) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:56,707::__init__::232::IOProcess::(_processLogs) DEBUG|(6945) Queuing response >Thread-16::DEBUG::2014-08-11 15:24:56,709::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:24:56,735::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00531213 s, 85.7 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:24:56,738::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:56,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:56,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:56,742::__init__::232::IOProcess::(_processLogs) (6946) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:24:56,744::__init__::232::IOProcess::(_processLogs) DEBUG|(6946) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:56,747::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:56,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:56,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:56,750::__init__::232::IOProcess::(_processLogs) (6947) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:56,751::__init__::232::IOProcess::(_processLogs) (6947) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:56,754::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:56,756::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:56,758::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:56,759::__init__::232::IOProcess::(_processLogs) (6948) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:56,760::__init__::232::IOProcess::(_processLogs) (6948) Queuing response >Thread-14::DEBUG::2014-08-11 15:24:56,763::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:24:56,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:24:56,765::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:24:56,766::__init__::232::IOProcess::(_processLogs) (6949) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:24:56,767::__init__::232::IOProcess::(_processLogs) (6949) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:56,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:56,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:24:56,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:24:58,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:24:58,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135531 s, 75.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:24:59,311::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:24:59,312::task::595::Storage.TaskManager.Task::(_updateState) Task=`068ad6f3-ccdb-4219-9bef-b4013d0ba1a5`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:24:59,312::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:24:59,325::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:59,325::task::1191::Storage.TaskManager.Task::(prepare) Task=`068ad6f3-ccdb-4219-9bef-b4013d0ba1a5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:24:59,325::task::595::Storage.TaskManager.Task::(_updateState) Task=`068ad6f3-ccdb-4219-9bef-b4013d0ba1a5`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:24:59,325::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:24:59,326::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:24:59,326::task::993::Storage.TaskManager.Task::(_decref) Task=`068ad6f3-ccdb-4219-9bef-b4013d0ba1a5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:24:59,337::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:24:59,337::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eacd878-7ab8-44e5-a93d-61597b7c8d4a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:24:59,338::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:24:59,339::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`709db028-92dc-498e-8749-a0c7c17ba95e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:24:59,339::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:24:59,339::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:24:59,339::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`709db028-92dc-498e-8749-a0c7c17ba95e`::Granted request >Thread-34::DEBUG::2014-08-11 15:24:59,340::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7eacd878-7ab8-44e5-a93d-61597b7c8d4a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:24:59,340::task::993::Storage.TaskManager.Task::(_decref) Task=`7eacd878-7ab8-44e5-a93d-61597b7c8d4a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:24:59,360::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:59,361::task::1191::Storage.TaskManager.Task::(prepare) Task=`7eacd878-7ab8-44e5-a93d-61597b7c8d4a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:24:59,361::task::595::Storage.TaskManager.Task::(_updateState) Task=`7eacd878-7ab8-44e5-a93d-61597b7c8d4a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:24:59,361::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:24:59,361::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:24:59,362::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:24:59,362::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:24:59,362::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:24:59,362::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:24:59,363::task::993::Storage.TaskManager.Task::(_decref) Task=`7eacd878-7ab8-44e5-a93d-61597b7c8d4a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:25:00,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:00,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133374 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:01,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:01,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:01,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:25:02,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:02,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280123 s, 36.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:04,970::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:05,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132297 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:06,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`d515b814-16ca-473a-8300-b7bcbb63a64d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:06,213::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:25:06,214::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00531213', 'lastCheck': '9.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:06,214::task::1191::Storage.TaskManager.Task::(prepare) Task=`d515b814-16ca-473a-8300-b7bcbb63a64d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00531213', 'lastCheck': '9.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:06,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`d515b814-16ca-473a-8300-b7bcbb63a64d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:06,214::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:06,215::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:06,215::task::993::Storage.TaskManager.Task::(_decref) Task=`d515b814-16ca-473a-8300-b7bcbb63a64d`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:25:06,237::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:25:06,766::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:06,766::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:06,802::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:25:06,836::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:06,837::__init__::232::IOProcess::(_processLogs) (6950) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:06,840::__init__::232::IOProcess::(_processLogs) DEBUG|(6950) Queuing response >Thread-16::DEBUG::2014-08-11 15:25:06,841::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:06,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:06,862::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:25:06,864::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000410432 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:06,865::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:06,866::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:06,866::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:06,866::__init__::232::IOProcess::(_processLogs) (6951) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:06,867::__init__::232::IOProcess::(_processLogs) DEBUG|(6951) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:06,868::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:06,869::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:06,870::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:06,870::__init__::232::IOProcess::(_processLogs) (6952) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:06,870::__init__::232::IOProcess::(_processLogs) (6952) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:06,870::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:06,871::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:06,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:06,872::__init__::232::IOProcess::(_processLogs) (6953) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:06,872::__init__::232::IOProcess::(_processLogs) (6953) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:06,872::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:06,873::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:06,873::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:06,874::__init__::232::IOProcess::(_processLogs) (6954) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:06,875::__init__::232::IOProcess::(_processLogs) (6954) Queuing response >Dummy-33::DEBUG::2014-08-11 15:25:07,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:07,059::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132292 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:09,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:09,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131935 s, 77.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:09,374::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:25:09,375::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6efa42a-1474-4da0-8056-5a1520ff5e03`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:09,376::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:25:09,388::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:09,389::task::1191::Storage.TaskManager.Task::(prepare) Task=`d6efa42a-1474-4da0-8056-5a1520ff5e03`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:09,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6efa42a-1474-4da0-8056-5a1520ff5e03`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:09,389::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:09,389::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:09,390::task::993::Storage.TaskManager.Task::(_decref) Task=`d6efa42a-1474-4da0-8056-5a1520ff5e03`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:25:09,400::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:25:09,400::task::595::Storage.TaskManager.Task::(_updateState) Task=`3483d470-fc8b-407e-b4f9-dcd1b1984e23`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:25:09,401::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:25:09,401::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3b53ddbe-34ee-43ad-b391-ff08ffa4a897`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:25:09,401::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:25:09,402::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:25:09,402::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3b53ddbe-34ee-43ad-b391-ff08ffa4a897`::Granted request >Thread-34::DEBUG::2014-08-11 15:25:09,403::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3483d470-fc8b-407e-b4f9-dcd1b1984e23`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:25:09,403::task::993::Storage.TaskManager.Task::(_decref) Task=`3483d470-fc8b-407e-b4f9-dcd1b1984e23`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:25:09,423::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:09,423::task::1191::Storage.TaskManager.Task::(prepare) Task=`3483d470-fc8b-407e-b4f9-dcd1b1984e23`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:09,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`3483d470-fc8b-407e-b4f9-dcd1b1984e23`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:25:09,423::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:25:09,424::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:25:09,424::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:25:09,424::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:25:09,425::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:25:09,425::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:25:09,425::task::993::Storage.TaskManager.Task::(_decref) Task=`3483d470-fc8b-407e-b4f9-dcd1b1984e23`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:25:11,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:11,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01323 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:11,871::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:11,872::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:11,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:25:13,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:13,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133646 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:15,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:15,281::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132562 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:16,875::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:16,876::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:16,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:25:16,936::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:16,937::__init__::232::IOProcess::(_processLogs) (6955) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:16,941::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:25:16,940::__init__::232::IOProcess::(_processLogs) DEBUG|(6955) Queuing response >Thread-16::DEBUG::2014-08-11 15:25:16,941::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:16,963::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:25:16,964::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000769565 s, 591 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:16,967::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:16,968::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:16,968::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:16,968::__init__::232::IOProcess::(_processLogs) (6956) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:16,969::__init__::232::IOProcess::(_processLogs) DEBUG|(6956) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:16,970::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:16,971::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:16,971::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:16,972::__init__::232::IOProcess::(_processLogs) (6957) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:16,972::__init__::232::IOProcess::(_processLogs) (6957) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:16,973::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:16,973::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:16,974::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:16,974::__init__::232::IOProcess::(_processLogs) (6958) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:16,974::__init__::232::IOProcess::(_processLogs) (6958) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:16,975::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:16,975::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:16,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:16,977::__init__::232::IOProcess::(_processLogs) (6959) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:16,977::__init__::232::IOProcess::(_processLogs) (6959) Queuing response >Dummy-33::DEBUG::2014-08-11 15:25:17,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:17,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0239551 s, 42.7 MB/s\n'; <rc> = 0 >Thread-55::DEBUG::2014-08-11 15:25:18,534::task::595::Storage.TaskManager.Task::(_updateState) Task=`7efebb36-be00-4573-83de-969022fc6fe4`::moving from state init -> state preparing >Thread-55::INFO::2014-08-11 15:25:18,534::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:25:18,535::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:18,536::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:18,536::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:18,536::__init__::232::IOProcess::(_processLogs) (6960) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:25:18,537::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6960) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:18,538::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:18,539::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-55::INFO::2014-08-11 15:25:18,539::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:25:18,540::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-55::DEBUG::2014-08-11 15:25:18,541::task::1191::Storage.TaskManager.Task::(prepare) Task=`7efebb36-be00-4573-83de-969022fc6fe4`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:25:18,541::__init__::232::IOProcess::(_processLogs) (6961) Got request for method 'stat' >Thread-55::DEBUG::2014-08-11 15:25:18,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`7efebb36-be00-4573-83de-969022fc6fe4`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:25:18,543::__init__::232::IOProcess::(_processLogs) (6961) Queuing response >Thread-55::DEBUG::2014-08-11 15:25:18,543::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-55::DEBUG::2014-08-11 15:25:18,544::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-55::DEBUG::2014-08-11 15:25:18,544::task::993::Storage.TaskManager.Task::(_decref) Task=`7efebb36-be00-4573-83de-969022fc6fe4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:25:19,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:19,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133979 s, 76.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:19,440::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:25:19,442::task::595::Storage.TaskManager.Task::(_updateState) Task=`d57d6c22-6864-401c-968b-f0393a21e15d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:19,443::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:25:19,457::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:19,458::task::1191::Storage.TaskManager.Task::(prepare) Task=`d57d6c22-6864-401c-968b-f0393a21e15d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:19,458::task::595::Storage.TaskManager.Task::(_updateState) Task=`d57d6c22-6864-401c-968b-f0393a21e15d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:19,458::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:19,458::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:19,459::task::993::Storage.TaskManager.Task::(_decref) Task=`d57d6c22-6864-401c-968b-f0393a21e15d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:25:19,468::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:25:19,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`19010791-2d43-441d-80a8-455110304093`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:25:19,469::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:25:19,470::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d761d47f-4635-4ef9-93e7-03cf3fdb18cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:25:19,470::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:25:19,471::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:25:19,471::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d761d47f-4635-4ef9-93e7-03cf3fdb18cb`::Granted request >Thread-34::DEBUG::2014-08-11 15:25:19,471::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`19010791-2d43-441d-80a8-455110304093`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:25:19,472::task::993::Storage.TaskManager.Task::(_decref) Task=`19010791-2d43-441d-80a8-455110304093`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:25:19,491::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:19,492::task::1191::Storage.TaskManager.Task::(prepare) Task=`19010791-2d43-441d-80a8-455110304093`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959490048', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:19,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`19010791-2d43-441d-80a8-455110304093`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:25:19,492::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:25:19,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:25:19,493::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:25:19,493::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:25:19,493::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:25:19,494::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:25:19,494::task::993::Storage.TaskManager.Task::(_decref) Task=`19010791-2d43-441d-80a8-455110304093`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:25:21,332::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf5f7f3a-1e94-4a9a-9958-bafa418a9f4e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:21,332::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:25:21,332::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000769565', 'lastCheck': '4.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:21,333::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf5f7f3a-1e94-4a9a-9958-bafa418a9f4e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000769565', 'lastCheck': '4.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:21,333::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf5f7f3a-1e94-4a9a-9958-bafa418a9f4e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:21,333::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:21,333::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:21,333::task::993::Storage.TaskManager.Task::(_decref) Task=`bf5f7f3a-1e94-4a9a-9958-bafa418a9f4e`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:25:21,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:25:21,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:21,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133153 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:21,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:21,971::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:22,036::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:25:23,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:23,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137355 s, 74.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:25,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:25,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131918 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:26,978::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:26,978::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:27,036::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:27,037::__init__::232::IOProcess::(_processLogs) (6962) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:27,037::__init__::232::IOProcess::(_processLogs) (6962) Queuing response >Thread-16::DEBUG::2014-08-11 15:25:27,038::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:27,057::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:25:27,059::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000483144 s, 942 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:27,062::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:27,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:25:27,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:27,065::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:27,067::__init__::232::IOProcess::(_processLogs) (6963) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:27,069::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6963) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:27,070::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:25:27,071::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:27,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:27,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:27,073::__init__::232::IOProcess::(_processLogs) (6964) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:27,074::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6964) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:27,074::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:27,075::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:27,075::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:27,075::__init__::232::IOProcess::(_processLogs) (6965) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:27,076::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6965) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:27,077::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:27,077::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:27,077::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:27,078::__init__::232::IOProcess::(_processLogs) (6966) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:27,078::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6966) Queuing response >Dummy-33::DEBUG::2014-08-11 15:25:27,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:27,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132165 s, 77.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:29,505::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:25:29,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`82621e66-9e33-46e6-8280-8ecabd3b6177`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:29,507::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:25:29,520::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:29,520::task::1191::Storage.TaskManager.Task::(prepare) Task=`82621e66-9e33-46e6-8280-8ecabd3b6177`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:29,521::task::595::Storage.TaskManager.Task::(_updateState) Task=`82621e66-9e33-46e6-8280-8ecabd3b6177`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:29,521::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:29,521::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:29,521::task::993::Storage.TaskManager.Task::(_decref) Task=`82621e66-9e33-46e6-8280-8ecabd3b6177`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:25:29,532::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:25:29,533::task::595::Storage.TaskManager.Task::(_updateState) Task=`94de8e5f-46fe-4921-80bc-9236dd408b04`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:25:29,533::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:25:29,534::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c3da24ad-a4c1-4d4c-aa9f-8080997af5b4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:25:29,534::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:25:29,535::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:25:29,535::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c3da24ad-a4c1-4d4c-aa9f-8080997af5b4`::Granted request >Thread-34::DEBUG::2014-08-11 15:25:29,535::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`94de8e5f-46fe-4921-80bc-9236dd408b04`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:25:29,536::task::993::Storage.TaskManager.Task::(_decref) Task=`94de8e5f-46fe-4921-80bc-9236dd408b04`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:25:29,556::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:29,556::task::1191::Storage.TaskManager.Task::(prepare) Task=`94de8e5f-46fe-4921-80bc-9236dd408b04`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:29,556::task::595::Storage.TaskManager.Task::(_updateState) Task=`94de8e5f-46fe-4921-80bc-9236dd408b04`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:25:29,556::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:25:29,557::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:25:29,557::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:25:29,557::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:25:29,558::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:25:29,558::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:25:29,558::task::993::Storage.TaskManager.Task::(_decref) Task=`94de8e5f-46fe-4921-80bc-9236dd408b04`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:25:29,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:29,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131466 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:31,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:31,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136601 s, 75.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:32,080::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:32,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:32,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:25:33,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:33,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133233 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:34,769::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmShutdown with ('e9a797ed-f982-4208-af2a-d6558ca6dfed', '30', 'System Administrator has initiated shutdown of this Virtual Machine. Virtual Machine is shutting down.') {} flowID [2f19a0d4] >Thread-115::DEBUG::2014-08-11 15:25:34,770::BindingXMLRPC::1134::vds::(wrapper) return vmShutdown with {'status': {'message': 'Machine shutting down', 'code': 0}} >Thread-119::DEBUG::2014-08-11 15:25:34,770::utils::1221::utils.CallbackChain::(run) Starting callback chain. >Thread-119::DEBUG::2014-08-11 15:25:34,772::utils::1190::utils.Callback::(__call__) Calling guestAgentCallback with args=() and kwargs={} >Thread-119::DEBUG::2014-08-11 15:25:34,773::guestagent::377::vm.Vm::(desktopShutdown) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::desktopShutdown called >Dummy-33::DEBUG::2014-08-11 15:25:35,841::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:35,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133455 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:36,544::task::595::Storage.TaskManager.Task::(_updateState) Task=`f291e29c-fef5-4cdc-a8fa-9fe50985845f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:36,544::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:25:36,545::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000483144', 'lastCheck': '9.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:36,545::task::1191::Storage.TaskManager.Task::(prepare) Task=`f291e29c-fef5-4cdc-a8fa-9fe50985845f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000483144', 'lastCheck': '9.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:36,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`f291e29c-fef5-4cdc-a8fa-9fe50985845f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:36,545::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:36,546::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:36,546::task::993::Storage.TaskManager.Task::(_decref) Task=`f291e29c-fef5-4cdc-a8fa-9fe50985845f`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:25:36,630::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:25:37,080::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:37,080::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:37,081::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:37,081::__init__::232::IOProcess::(_processLogs) (6967) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:37,081::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6967) Queuing response >Thread-16::DEBUG::2014-08-11 15:25:37,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:25:37,101::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418191 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:37,102::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:37,137::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:37,138::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:37,138::__init__::232::IOProcess::(_processLogs) (6968) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:37,139::__init__::232::IOProcess::(_processLogs) (6968) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:37,139::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:37,139::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:37,140::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:37,140::__init__::232::IOProcess::(_processLogs) (6969) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:37,140::__init__::232::IOProcess::(_processLogs) (6969) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:37,141::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:37,141::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:37,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:37,142::__init__::232::IOProcess::(_processLogs) (6970) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:37,142::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6970) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:37,143::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:37,144::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:37,144::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:37,145::__init__::232::IOProcess::(_processLogs) (6971) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:37,145::__init__::232::IOProcess::(_processLogs) (6971) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:37,150::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:37,151::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:37,152::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:25:37,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:37,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132268 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:39,571::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:25:39,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`24af0e3c-f13a-40b5-8274-aa557d731bd0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:39,572::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:25:39,585::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:39,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`24af0e3c-f13a-40b5-8274-aa557d731bd0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:39,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`24af0e3c-f13a-40b5-8274-aa557d731bd0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:39,585::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:39,586::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:39,586::task::993::Storage.TaskManager.Task::(_decref) Task=`24af0e3c-f13a-40b5-8274-aa557d731bd0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:25:39,597::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:25:39,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`f84693ae-b61b-4e75-b6b0-7dbc507b71cc`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:25:39,598::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:25:39,599::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2317db90-7440-44c5-8438-19192ee49735`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:25:39,599::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:25:39,599::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:25:39,599::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2317db90-7440-44c5-8438-19192ee49735`::Granted request >Thread-34::DEBUG::2014-08-11 15:25:39,600::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f84693ae-b61b-4e75-b6b0-7dbc507b71cc`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:25:39,600::task::993::Storage.TaskManager.Task::(_decref) Task=`f84693ae-b61b-4e75-b6b0-7dbc507b71cc`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:25:39,620::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:39,620::task::1191::Storage.TaskManager.Task::(prepare) Task=`f84693ae-b61b-4e75-b6b0-7dbc507b71cc`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:39,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`f84693ae-b61b-4e75-b6b0-7dbc507b71cc`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:25:39,621::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:25:39,621::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:25:39,621::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:25:39,622::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:25:39,622::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:25:39,622::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:25:39,622::task::993::Storage.TaskManager.Task::(_decref) Task=`f84693ae-b61b-4e75-b6b0-7dbc507b71cc`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:25:39,712::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:25:39,713::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:25:39,713::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '16', 'acpiEnable': 'true', 'guestFQDN': u'localhost.localdomain', 'pid': '20793', 'session': 'Unknown', 'netIfaces': [{u'hw': u'00:1a:4a:16:88:5c', u'name': u'eth0', u'inet': [u'10.35.128.92'], u'inet6': [u'fe80::21a:4aff:fe16:885c']}], 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'/', u'total': '4792987648', u'used': '874905600', u'fs': u'ext4'}, {u'path': u'/boot', u'total': '507744256', u'used': '34277376', u'fs': u'ext4'}, {u'path': u'/home', u'total': '458985472', u'used': '10764288', u'fs': u'ext4'}], 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '1709', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'localhost.localdomain', 'vmJobs': {}, 'elapsedTime': '9884', 'memoryStats': {u'swap_out': '0', u'majflt': '0', u'swap_usage': '0', u'swap_total': '4751352', u'swap_in': '0', u'mem_free': '877644', u'pageflt': '477', u'mem_total': '1000596', u'mem_unused': '808156'}, 'cpuSys': '0.47', 'appsList': [u'rhevm-guest-agent-common-1.0.9-5.el6ev', u'kernel-2.6.32-431.1.2.el6'], 'guestOs': u'2.6.32-431.1.2.el6.x86_64', 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'guestCPUCount': 1, 'vmType': 'kvm', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 1000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '4537640', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '82035', 'readRate': '0.00', 'truesize': '1169743872', 'writeRate': '272.54'}, u'hdc': {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '1.10', 'username': u'None', 'cpuUser': '1.88', 'status': 'Powering down', 'lastLogin': 1407750126.124831, 'guestIPs': u'10.35.128.92', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:25:39,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:39,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131112 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:42,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:42,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131201 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:42,159::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:42,161::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:25:42,237::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:25:42,743::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:25:42,744::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:25:42,745::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '16', 'acpiEnable': 'true', 'guestFQDN': u'localhost.localdomain', 'pid': '20793', 'session': 'Unknown', 'netIfaces': [{u'hw': u'00:1a:4a:16:88:5c', u'name': u'eth0', u'inet': [u'10.35.128.92'], u'inet6': [u'fe80::21a:4aff:fe16:885c']}], 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'disksUsage': [{u'path': u'/', u'total': '4792987648', u'used': '874905600', u'fs': u'ext4'}, {u'path': u'/boot', u'total': '507744256', u'used': '34277376', u'fs': u'ext4'}, {u'path': u'/home', u'total': '458985472', u'used': '10764288', u'fs': u'ext4'}], 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '1709', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'displayType': 'qxl', 'guestName': u'localhost.localdomain', 'vmJobs': {}, 'elapsedTime': '9887', 'memoryStats': {u'swap_out': '0', u'majflt': '0', u'swap_usage': '0', u'swap_total': '4751352', u'swap_in': '0', u'mem_free': '877660', u'pageflt': '15', u'mem_total': '1000596', u'mem_unused': '808164'}, 'cpuSys': '0.47', 'appsList': [u'rhevm-guest-agent-common-1.0.9-5.el6ev', u'kernel-2.6.32-431.1.2.el6'], 'guestOs': u'2.6.32-431.1.2.el6.x86_64', 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'guestCPUCount': 1, 'vmType': 'kvm', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 1000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '4537640', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '82035', 'readRate': '0.00', 'truesize': '1169743872', 'writeRate': '272.54'}, u'hdc': {'readLatency': '0', 'apparentsize': '0', 'writeLatency': '0', 'flushLatency': '0', 'readRate': '0.00', 'truesize': '0', 'writeRate': '0.00'}}, 'monitorResponse': '0', 'statsAge': '0.13', 'username': u'None', 'cpuUser': '1.88', 'status': 'Powering down', 'lastLogin': 1407750126.124831, 'guestIPs': u'10.35.128.92', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:25:44,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:44,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133177 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:44,260::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmDestroy with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} flowID [649e8048] >Thread-115::INFO::2014-08-11 15:25:44,261::API::333::vds::(destroy) vmContainerLock acquired by vm e9a797ed-f982-4208-af2a-d6558ca6dfed >Thread-115::DEBUG::2014-08-11 15:25:44,261::vm::4813::vm.Vm::(destroy) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::destroy Called >Thread-115::INFO::2014-08-11 15:25:44,262::vm::4746::vm.Vm::(releaseVm) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Release VM resources >Thread-115::DEBUG::2014-08-11 15:25:44,282::sampling::406::vm.Vm::(stop) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Stop statistics collection >Thread-55::DEBUG::2014-08-11 15:25:44,282::sampling::437::vm.Vm::(run) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Stats thread finished >Thread-115::DEBUG::2014-08-11 15:25:44,282::vmchannels::210::vds::(unregister) Delete fileno 50 from listener. >VM Channels Listener::DEBUG::2014-08-11 15:25:44,395::vmchannels::117::vds::(_do_del_channels) fileno 50 was removed from listener. >Thread-115::DEBUG::2014-08-11 15:25:44,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ea5e2a0-684f-461e-ab2d-ff2147700679`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:44,605::logUtils::44::dispatcher::(wrapper) Run and protect: teardownImage(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID=None) >libvirtEventLoop::DEBUG::2014-08-11 15:25:44,606::vm::5350::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::event Stopped detail 1 opaque None >Thread-115::DEBUG::2014-08-11 15:25:44,607::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a`ReqID=`df0dafc6-cd3f-4c42-afc5-5290b7f65609`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3252' at 'teardownImage' >libvirtEventLoop::INFO::2014-08-11 15:25:44,607::vm::2327::vm.Vm::(_onQemuDeath) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::underlying process disconnected >Thread-115::DEBUG::2014-08-11 15:25:44,608::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' for lock type 'shared' >Thread-115::DEBUG::2014-08-11 15:25:44,609::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' is free. Now locking as 'shared' (1 active user) >Thread-115::DEBUG::2014-08-11 15:25:44,609::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a`ReqID=`df0dafc6-cd3f-4c42-afc5-5290b7f65609`::Granted request >Thread-115::DEBUG::2014-08-11 15:25:44,610::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3ea5e2a0-684f-461e-ab2d-ff2147700679`::_resourcesAcquired: Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a (shared) >Thread-115::DEBUG::2014-08-11 15:25:44,610::task::993::Storage.TaskManager.Task::(_decref) Task=`3ea5e2a0-684f-461e-ab2d-ff2147700679`::ref 1 aborting False >Thread-115::INFO::2014-08-11 15:25:44,610::logUtils::47::dispatcher::(wrapper) Run and protect: teardownImage, Return response: None >Thread-115::DEBUG::2014-08-11 15:25:44,610::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ea5e2a0-684f-461e-ab2d-ff2147700679`::finished: None >Thread-115::DEBUG::2014-08-11 15:25:44,611::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ea5e2a0-684f-461e-ab2d-ff2147700679`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:44,611::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': < ResourceRef 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', isValid: 'True' obj: 'None'>} >Thread-115::DEBUG::2014-08-11 15:25:44,611::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:44,611::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' >Thread-115::DEBUG::2014-08-11 15:25:44,612::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' (0 active users) >Thread-115::DEBUG::2014-08-11 15:25:44,612::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' is free, finding out if anyone is waiting for it. >Thread-115::DEBUG::2014-08-11 15:25:44,612::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', Clearing records. >Thread-115::DEBUG::2014-08-11 15:25:44,612::task::993::Storage.TaskManager.Task::(_decref) Task=`3ea5e2a0-684f-461e-ab2d-ff2147700679`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:25:44,614::task::595::Storage.TaskManager.Task::(_updateState) Task=`a93db5a4-3860-43ea-ae9d-72cde322e037`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:44,614::logUtils::44::dispatcher::(wrapper) Run and protect: inappropriateDevices(thiefId='e9a797ed-f982-4208-af2a-d6558ca6dfed') >Thread-115::INFO::2014-08-11 15:25:44,616::logUtils::47::dispatcher::(wrapper) Run and protect: inappropriateDevices, Return response: None >Thread-115::DEBUG::2014-08-11 15:25:44,616::task::1191::Storage.TaskManager.Task::(prepare) Task=`a93db5a4-3860-43ea-ae9d-72cde322e037`::finished: None >Thread-115::DEBUG::2014-08-11 15:25:44,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`a93db5a4-3860-43ea-ae9d-72cde322e037`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:44,617::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:44,618::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:44,618::task::993::Storage.TaskManager.Task::(_decref) Task=`a93db5a4-3860-43ea-ae9d-72cde322e037`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:25:44,619::vm::4807::vm.Vm::(deleteVm) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Total desktops after destroy of e9a797ed-f982-4208-af2a-d6558ca6dfed is 0 >libvirtEventLoop::DEBUG::2014-08-11 15:25:44,619::vm::2771::vm.Vm::(setDownStatus) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Changed state to Down: Admin shut down from the engine (code=6) >Thread-115::DEBUG::2014-08-11 15:25:44,621::BindingXMLRPC::1134::vds::(wrapper) return vmDestroy with {'status': {'message': 'Machine destroyed', 'code': 0}} >libvirtEventLoop::DEBUG::2014-08-11 15:25:44,621::sampling::406::vm.Vm::(stop) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Stop statistics collection >Thread-119::DEBUG::2014-08-11 15:25:44,623::utils::1226::utils.CallbackChain::(run) Succeeded after invoking guestAgentCallback >Dummy-33::DEBUG::2014-08-11 15:25:46,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:46,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131997 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:47,146::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:47,147::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:47,147::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:47,147::__init__::232::IOProcess::(_processLogs) (6972) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:47,148::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6972) Queuing response >Thread-16::DEBUG::2014-08-11 15:25:47,149::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:25:47,167::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417051 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:47,169::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:47,237::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:47,238::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:47,238::__init__::232::IOProcess::(_processLogs) (6973) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:47,239::__init__::232::IOProcess::(_processLogs) (6973) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:47,239::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:47,239::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:47,240::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:47,240::__init__::232::IOProcess::(_processLogs) (6974) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:47,240::__init__::232::IOProcess::(_processLogs) (6974) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:47,241::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:47,241::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:47,242::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:47,244::__init__::232::IOProcess::(_processLogs) (6975) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:47,244::__init__::232::IOProcess::(_processLogs) (6975) Queuing response >Thread-14::DEBUG::2014-08-11 15:25:47,245::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:47,246::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:47,246::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:47,246::__init__::232::IOProcess::(_processLogs) (6976) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:47,247::__init__::232::IOProcess::(_processLogs) (6976) Queuing response >Dummy-33::DEBUG::2014-08-11 15:25:48,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:48,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132177 s, 77.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:49,639::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:25:49,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`084fbd5b-820f-4d43-8d9b-1d7dea0ca425`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:49,640::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:25:49,653::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:49,653::task::1191::Storage.TaskManager.Task::(prepare) Task=`084fbd5b-820f-4d43-8d9b-1d7dea0ca425`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:49,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`084fbd5b-820f-4d43-8d9b-1d7dea0ca425`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:49,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:49,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:49,654::task::993::Storage.TaskManager.Task::(_decref) Task=`084fbd5b-820f-4d43-8d9b-1d7dea0ca425`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:25:49,666::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:25:49,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1a68ffa-8402-4fe7-bebb-cd1b20612cf7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:25:49,667::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:25:49,668::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5925ee99-3966-4985-89e3-5ef5da89894d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:25:49,668::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:25:49,670::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:25:49,670::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5925ee99-3966-4985-89e3-5ef5da89894d`::Granted request >Thread-34::DEBUG::2014-08-11 15:25:49,671::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a1a68ffa-8402-4fe7-bebb-cd1b20612cf7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:25:49,671::task::993::Storage.TaskManager.Task::(_decref) Task=`a1a68ffa-8402-4fe7-bebb-cd1b20612cf7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:25:49,691::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:49,691::task::1191::Storage.TaskManager.Task::(prepare) Task=`a1a68ffa-8402-4fe7-bebb-cd1b20612cf7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:49,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`a1a68ffa-8402-4fe7-bebb-cd1b20612cf7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:25:49,692::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:25:49,692::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:25:49,692::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:25:49,693::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:25:49,693::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:25:49,693::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:25:49,693::task::993::Storage.TaskManager.Task::(_decref) Task=`a1a68ffa-8402-4fe7-bebb-cd1b20612cf7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:25:50,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:50,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199722 s, 51.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:51,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`0df3e21a-8cd6-43a9-bfd9-3ddc8ba819da`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:51,789::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:25:51,789::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417051', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:51,789::task::1191::Storage.TaskManager.Task::(prepare) Task=`0df3e21a-8cd6-43a9-bfd9-3ddc8ba819da`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417051', 'lastCheck': '4.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:25:51,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`0df3e21a-8cd6-43a9-bfd9-3ddc8ba819da`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:51,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:51,790::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:51,790::task::993::Storage.TaskManager.Task::(_decref) Task=`0df3e21a-8cd6-43a9-bfd9-3ddc8ba819da`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:25:52,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:52,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132123 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:54,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:54,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013236 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:25:56,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:56,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137609 s, 74.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:57,248::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:57,249::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:57,251::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:25:57,250::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:25:57,252::__init__::232::IOProcess::(_processLogs) (6977) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:57,269::__init__::232::IOProcess::(_processLogs) (6977) Queuing response >Thread-16::DEBUG::2014-08-11 15:25:57,270::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000534682 s, 851 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:25:57,271::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:25:57,272::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:25:57,272::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:57,272::__init__::232::IOProcess::(_processLogs) (6978) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:25:57,273::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:57,274::__init__::232::IOProcess::(_processLogs) (6979) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:57,274::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:57,275::__init__::232::IOProcess::(_processLogs) (6980) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:57,276::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:25:57,276::__init__::232::IOProcess::(_processLogs) (6981) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:25:57,276::__init__::232::IOProcess::(_processLogs) (6981) Queuing response >Dummy-33::DEBUG::2014-08-11 15:25:58,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:25:58,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132529 s, 77.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:25:59,732::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:25:59,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bd0ae84-1ad6-4fae-92e6-65fff755ee1e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:25:59,734::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:25:59,746::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:59,747::task::1191::Storage.TaskManager.Task::(prepare) Task=`6bd0ae84-1ad6-4fae-92e6-65fff755ee1e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:25:59,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bd0ae84-1ad6-4fae-92e6-65fff755ee1e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:25:59,747::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:25:59,747::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:25:59,748::task::993::Storage.TaskManager.Task::(_decref) Task=`6bd0ae84-1ad6-4fae-92e6-65fff755ee1e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:25:59,758::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:25:59,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`41d633d2-e9cc-43d6-bb80-625ba36dc7bf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:25:59,759::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:25:59,760::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`336b440d-7e28-40e0-8560-6aaf87573e9b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:25:59,760::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:25:59,760::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:25:59,761::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`336b440d-7e28-40e0-8560-6aaf87573e9b`::Granted request >Thread-34::DEBUG::2014-08-11 15:25:59,761::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`41d633d2-e9cc-43d6-bb80-625ba36dc7bf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:25:59,761::task::993::Storage.TaskManager.Task::(_decref) Task=`41d633d2-e9cc-43d6-bb80-625ba36dc7bf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:25:59,781::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:59,782::task::1191::Storage.TaskManager.Task::(prepare) Task=`41d633d2-e9cc-43d6-bb80-625ba36dc7bf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:25:59,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`41d633d2-e9cc-43d6-bb80-625ba36dc7bf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:25:59,782::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:25:59,782::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:25:59,783::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:25:59,783::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:25:59,783::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:25:59,783::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:25:59,784::task::993::Storage.TaskManager.Task::(_decref) Task=`41d633d2-e9cc-43d6-bb80-625ba36dc7bf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:26:00,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:00,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133582 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:26:02,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:02,626::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0266666 s, 38.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:26:04,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:04,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137367 s, 74.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:06,883::task::595::Storage.TaskManager.Task::(_updateState) Task=`11cea450-1a61-4155-a996-ec8afc2ac44b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:06,884::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:26:06,884::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000534682', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:06,885::task::1191::Storage.TaskManager.Task::(prepare) Task=`11cea450-1a61-4155-a996-ec8afc2ac44b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000534682', 'lastCheck': '9.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:06,885::task::595::Storage.TaskManager.Task::(_updateState) Task=`11cea450-1a61-4155-a996-ec8afc2ac44b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:06,885::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:06,885::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:06,885::task::993::Storage.TaskManager.Task::(_decref) Task=`11cea450-1a61-4155-a996-ec8afc2ac44b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:26:06,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:06,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132316 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:07,278::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:07,278::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:07,279::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:07,279::__init__::232::IOProcess::(_processLogs) (6982) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:07,280::__init__::232::IOProcess::(_processLogs) DEBUG|(6982) Queuing response >Thread-16::DEBUG::2014-08-11 15:26:07,281::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:26:07,299::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428237 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:07,300::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:07,301::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:07,301::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:07,301::__init__::232::IOProcess::(_processLogs) (6983) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:07,302::__init__::232::IOProcess::(_processLogs) DEBUG|(6983) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:07,303::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:07,303::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:07,303::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:07,304::__init__::232::IOProcess::(_processLogs) DEBUG|(6984) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:07,304::__init__::232::IOProcess::(_processLogs) (6984) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:07,305::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:07,305::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:07,305::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:07,306::__init__::232::IOProcess::(_processLogs) DEBUG|(6985) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:07,306::__init__::232::IOProcess::(_processLogs) (6985) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:07,307::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:07,307::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:07,307::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:07,307::__init__::232::IOProcess::(_processLogs) DEBUG|(6986) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:07,308::__init__::232::IOProcess::(_processLogs) (6986) Queuing response >Dummy-33::DEBUG::2014-08-11 15:26:08,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:08,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132617 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:09,796::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:26:09,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5b86fde-0bd9-421d-b5ba-9fb8cec96e85`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:09,797::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:26:09,810::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:09,810::task::1191::Storage.TaskManager.Task::(prepare) Task=`c5b86fde-0bd9-421d-b5ba-9fb8cec96e85`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:09,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5b86fde-0bd9-421d-b5ba-9fb8cec96e85`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:09,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:09,811::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:09,811::task::993::Storage.TaskManager.Task::(_decref) Task=`c5b86fde-0bd9-421d-b5ba-9fb8cec96e85`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:26:09,822::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:26:09,822::task::595::Storage.TaskManager.Task::(_updateState) Task=`c31f0f9b-bc23-44af-a6e9-57347a82ac72`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:26:09,823::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:26:09,824::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`94437f0d-9d58-481f-bf07-8c485ac717c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:26:09,824::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:26:09,824::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:26:09,824::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`94437f0d-9d58-481f-bf07-8c485ac717c8`::Granted request >Thread-34::DEBUG::2014-08-11 15:26:09,825::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c31f0f9b-bc23-44af-a6e9-57347a82ac72`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:26:09,825::task::993::Storage.TaskManager.Task::(_decref) Task=`c31f0f9b-bc23-44af-a6e9-57347a82ac72`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:26:09,845::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:09,845::task::1191::Storage.TaskManager.Task::(prepare) Task=`c31f0f9b-bc23-44af-a6e9-57347a82ac72`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:09,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`c31f0f9b-bc23-44af-a6e9-57347a82ac72`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:26:09,846::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:26:09,846::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:26:09,846::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:26:09,847::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:26:09,847::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:26:09,847::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:26:09,847::task::993::Storage.TaskManager.Task::(_decref) Task=`c31f0f9b-bc23-44af-a6e9-57347a82ac72`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:26:11,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:11,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132263 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:26:13,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:13,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132442 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:26:15,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:15,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131952 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:26:17,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:17,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132034 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:17,311::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:17,311::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:17,311::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:17,312::__init__::232::IOProcess::(_processLogs) (6987) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:17,312::__init__::232::IOProcess::(_processLogs) (6987) Queuing response >Thread-16::DEBUG::2014-08-11 15:26:17,313::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:26:17,330::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420796 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:17,332::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:17,332::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:17,332::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:17,333::__init__::232::IOProcess::(_processLogs) (6988) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:17,334::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:17,334::__init__::232::IOProcess::(_processLogs) (6989) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:17,335::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:17,335::__init__::232::IOProcess::(_processLogs) (6990) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:17,336::__init__::232::IOProcess::(_processLogs) (6990) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:17,337::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:17,337::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:17,338::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:17,339::__init__::232::IOProcess::(_processLogs) (6991) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:17,339::__init__::232::IOProcess::(_processLogs) (6991) Queuing response >Dummy-33::DEBUG::2014-08-11 15:26:19,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:19,267::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013268 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:19,860::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:26:19,861::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bfe5bfb-0d53-4cd4-9eaa-9bde7483837a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:19,861::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:26:19,874::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:19,874::task::1191::Storage.TaskManager.Task::(prepare) Task=`6bfe5bfb-0d53-4cd4-9eaa-9bde7483837a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:19,875::task::595::Storage.TaskManager.Task::(_updateState) Task=`6bfe5bfb-0d53-4cd4-9eaa-9bde7483837a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:19,875::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:19,875::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:19,875::task::993::Storage.TaskManager.Task::(_decref) Task=`6bfe5bfb-0d53-4cd4-9eaa-9bde7483837a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:26:19,886::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:26:19,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5b16580-0f03-4a4d-b8a6-d5b29d891205`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:26:19,887::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:26:19,887::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c3c57c07-00f1-4bc8-9220-6fb0ff248305`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:26:19,888::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:26:19,888::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:26:19,888::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c3c57c07-00f1-4bc8-9220-6fb0ff248305`::Granted request >Thread-34::DEBUG::2014-08-11 15:26:19,889::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c5b16580-0f03-4a4d-b8a6-d5b29d891205`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:26:19,889::task::993::Storage.TaskManager.Task::(_decref) Task=`c5b16580-0f03-4a4d-b8a6-d5b29d891205`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:26:19,909::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:19,909::task::1191::Storage.TaskManager.Task::(prepare) Task=`c5b16580-0f03-4a4d-b8a6-d5b29d891205`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:19,909::task::595::Storage.TaskManager.Task::(_updateState) Task=`c5b16580-0f03-4a4d-b8a6-d5b29d891205`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:26:19,910::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:26:19,910::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:26:19,910::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:26:19,910::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:26:19,911::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:26:19,911::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:26:19,911::task::993::Storage.TaskManager.Task::(_decref) Task=`c5b16580-0f03-4a4d-b8a6-d5b29d891205`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:26:21,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:21,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171251 s, 59.8 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:21,934::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmCreate with ({'acpiEnable': 'true', 'emulatedMachine': 'rhel6.5.0', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'memGuaranteedSize': 1024, 'transparentHugePages': 'true', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Conroe', 'smp': '1', 'numaTune': {'mode': 'preferred'}, 'custom': {'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebbdevice_d94c2e40-01d5-4ee5-aa30-3e337a45b0a1device_a5ecb96c-a652-4bf9-8db6-c4a2f38f3d90': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=a5ecb96c-a652-4bf9-8db6-c4a2f38f3d90, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=b43f7f69-f2c6-42da-8c56-d6f4edc5c110, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebbdevice_d94c2e40-01d5-4ee5-aa30-3e337a45b0a1': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=d94c2e40-01d5-4ee5-aa30-3e337a45b0a1, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebb': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=aeb94a0e-c203-4e6d-9efd-08177faaaebb, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebbdevice_d94c2e40-01d5-4ee5-aa30-3e337a45b0a1device_a5ecb96c-a652-4bf9-8db6-c4a2f38f3d90device_2ad8909e-5f19-4632-893d-e09633c77389': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=2ad8909e-5f19-4632-893d-e09633c77389, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}'}, 'vmType': 'kvm', 'memSize': 1024, 'smpCoresPerSocket': '1', 'vmName': 'on-qa', 'nice': '0', 'bootMenuEnable': 'false', 'copyPasteEnable': 'true', 'smartcardEnable': 'false', 'fileTransferEnable': 'true', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'displayNetwork': 'ovirtmgmt', 'devices': [{'device': 'qxl', 'specParams': {'vram': '32768', 'ram': '65536', 'heads': '1'}, 'type': 'video', 'deviceId': 'ff801460-6df8-4aa2-a228-0c45c6c38ad8', 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}}, {'index': '2', 'iface': 'ide', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'specParams': {'path': ''}, 'readonly': 'true', 'deviceId': '4cd4b91b-61ce-4d2c-9ab0-78375c9e7810', 'path': '', 'device': 'cdrom', 'shared': 'false', 'type': 'disk'}, {'index': 0, 'iface': 'virtio', 'format': 'raw', 'optional': 'false', 'poolID': '8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'volumeID': 'd69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'specParams': {}, 'readonly': 'false', 'domainID': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'deviceId': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'false', 'propagateErrors': 'off', 'type': 'disk'}, {'nicModel': 'pv', 'macAddr': '00:1a:4a:16:88:5c', 'linkActive': 'true', 'network': 'ovirtmgmt', 'bootOrder': '1', 'custom': {'queues': '7'}, 'filter': 'vdsm-no-mac-spoofing', 'specParams': {'inbound': {}, 'outbound': {}}, 'deviceId': '39207364-c022-4423-a8da-8304ca192523', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'index': '0', 'specParams': {}, 'deviceId': 'f19a6f0a-147c-48d7-a88a-eb31eb526d1e', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}], 'timeOffset': '0', 'maxVCpus': '160', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard', 'display': 'qxl'},) {} flowID [176706ca] >Thread-115::INFO::2014-08-11 15:26:21,935::clientIF::391::vds::(createVm) vmContainerLock acquired by vm e9a797ed-f982-4208-af2a-d6558ca6dfed >Thread-115::DEBUG::2014-08-11 15:26:21,943::clientIF::404::vds::(createVm) Total desktops after creation of e9a797ed-f982-4208-af2a-d6558ca6dfed is 1 >Thread-120::DEBUG::2014-08-11 15:26:21,943::vm::2228::vm.Vm::(_startUnderlyingVm) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Start >Thread-115::DEBUG::2014-08-11 15:26:21,945::BindingXMLRPC::1134::vds::(wrapper) return vmCreate with {'status': {'message': 'Done', 'code': 0}, 'vmList': {'acpiEnable': 'true', 'emulatedMachine': 'rhel6.5.0', 'pid': '0', 'memGuaranteedSize': 1024, 'transparentHugePages': 'true', 'displaySecurePort': '-1', 'spiceSslCipherSuite': 'DEFAULT', 'cpuType': 'Conroe', 'smp': '1', 'numaTune': {'mode': 'preferred'}, 'custom': {'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebbdevice_d94c2e40-01d5-4ee5-aa30-3e337a45b0a1device_a5ecb96c-a652-4bf9-8db6-c4a2f38f3d90': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=a5ecb96c-a652-4bf9-8db6-c4a2f38f3d90, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=b43f7f69-f2c6-42da-8c56-d6f4edc5c110, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebbdevice_d94c2e40-01d5-4ee5-aa30-3e337a45b0a1': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=d94c2e40-01d5-4ee5-aa30-3e337a45b0a1, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebb': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=aeb94a0e-c203-4e6d-9efd-08177faaaebb, device=virtio-serial, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, managed=false, plugged=true, readOnly=false, deviceAlias=virtio-serial0, customProperties={}, snapshotId=null}', 'device_b43f7f69-f2c6-42da-8c56-d6f4edc5c110device_aeb94a0e-c203-4e6d-9efd-08177faaaebbdevice_d94c2e40-01d5-4ee5-aa30-3e337a45b0a1device_a5ecb96c-a652-4bf9-8db6-c4a2f38f3d90device_2ad8909e-5f19-4632-893d-e09633c77389': 'VmDevice {vmId=e9a797ed-f982-4208-af2a-d6558ca6dfed, deviceId=2ad8909e-5f19-4632-893d-e09633c77389, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}'}, 'vmType': 'kvm', 'memSize': 1024, 'smpCoresPerSocket': '1', 'vmName': 'on-qa', 'nice': '0', 'status': 'WaitForLaunch', 'bootMenuEnable': 'false', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'copyPasteEnable': 'true', 'displayIp': '10.35.128.39', 'displayPort': '-1', 'smartcardEnable': 'false', 'spiceSecureChannels': 'smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard', 'fileTransferEnable': 'true', 'nicModel': 'rtl8139,pv', 'keyboardLayout': 'en-us', 'kvmEnable': 'true', 'pitReinjection': 'false', 'displayNetwork': 'ovirtmgmt', 'devices': [{'device': 'qxl', 'specParams': {'vram': '32768', 'ram': '65536', 'heads': '1'}, 'type': 'video', 'deviceId': 'ff801460-6df8-4aa2-a228-0c45c6c38ad8', 'address': {'slot': '0x02', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}}, {'index': '2', 'iface': 'ide', 'address': {'bus': '1', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'specParams': {'path': ''}, 'readonly': 'true', 'deviceId': '4cd4b91b-61ce-4d2c-9ab0-78375c9e7810', 'path': '', 'device': 'cdrom', 'shared': 'false', 'type': 'disk'}, {'index': 0, 'iface': 'virtio', 'format': 'raw', 'optional': 'false', 'poolID': '8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'volumeID': 'd69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'specParams': {}, 'readonly': 'false', 'domainID': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'deviceId': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'address': {'slot': '0x06', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'disk', 'shared': 'false', 'propagateErrors': 'off', 'type': 'disk'}, {'nicModel': 'pv', 'macAddr': '00:1a:4a:16:88:5c', 'linkActive': 'true', 'network': 'ovirtmgmt', 'bootOrder': '1', 'custom': {'queues': '7'}, 'filter': 'vdsm-no-mac-spoofing', 'specParams': {'inbound': {}, 'outbound': {}}, 'deviceId': '39207364-c022-4423-a8da-8304ca192523', 'address': {'slot': '0x03', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'bridge', 'type': 'interface'}, {'index': '0', 'specParams': {}, 'deviceId': 'f19a6f0a-147c-48d7-a88a-eb31eb526d1e', 'address': {'slot': '0x04', 'bus': '0x00', 'domain': '0x0000', 'type': 'pci', 'function': '0x0'}, 'device': 'scsi', 'model': 'virtio-scsi', 'type': 'controller'}], 'timeOffset': '0', 'maxVCpus': '160', 'clientIp': '', 'display': 'qxl'}} >Thread-120::DEBUG::2014-08-11 15:26:21,945::vm::2232::vm.Vm::(_startUnderlyingVm) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::_ongoingCreations acquired >Thread-120::INFO::2014-08-11 15:26:21,947::vm::3224::vm.Vm::(_run) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::VM wrapper has started >Thread-120::DEBUG::2014-08-11 15:26:21,950::task::595::Storage.TaskManager.Task::(_updateState) Task=`778475e7-208f-434f-8fcc-91bc16440e45`::moving from state init -> state preparing >Thread-120::INFO::2014-08-11 15:26:21,952::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:26:21,955::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:21,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:21,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:21,961::__init__::232::IOProcess::(_processLogs) DEBUG|(6992) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:26:21,963::__init__::232::IOProcess::(_processLogs) (6992) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:21,965::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-120::INFO::2014-08-11 15:26:21,966::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:26:21,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-120::DEBUG::2014-08-11 15:26:21,967::task::1191::Storage.TaskManager.Task::(prepare) Task=`778475e7-208f-434f-8fcc-91bc16440e45`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:26:21,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-120::DEBUG::2014-08-11 15:26:21,968::task::595::Storage.TaskManager.Task::(_updateState) Task=`778475e7-208f-434f-8fcc-91bc16440e45`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:26:21,969::__init__::232::IOProcess::(_processLogs) (6993) Got request for method 'stat' >Thread-120::DEBUG::2014-08-11 15:26:21,969::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:26:21,970::__init__::232::IOProcess::(_processLogs) (6993) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:21,971::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-120::DEBUG::2014-08-11 15:26:21,972::task::993::Storage.TaskManager.Task::(_decref) Task=`778475e7-208f-434f-8fcc-91bc16440e45`::ref 0 aborting False >Thread-120::INFO::2014-08-11 15:26:21,972::clientIF::332::vds::(prepareVolumePath) prepared volume path: >Thread-120::DEBUG::2014-08-11 15:26:21,973::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4c0b0d2-9e6e-4350-91ca-61724daaee0a`::moving from state init -> state preparing >Thread-120::INFO::2014-08-11 15:26:21,973::logUtils::44::dispatcher::(wrapper) Run and protect: prepareImage(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', leafUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861') >Thread-120::DEBUG::2014-08-11 15:26:21,973::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a`ReqID=`84f87e8e-ec4d-4c46-8ec6-14f525631d5a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3204' at 'prepareImage' >Thread-120::DEBUG::2014-08-11 15:26:21,974::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' for lock type 'shared' >Thread-120::DEBUG::2014-08-11 15:26:21,974::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' is free. Now locking as 'shared' (1 active user) >Thread-120::DEBUG::2014-08-11 15:26:21,974::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a`ReqID=`84f87e8e-ec4d-4c46-8ec6-14f525631d5a`::Granted request >Thread-120::DEBUG::2014-08-11 15:26:21,975::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f4c0b0d2-9e6e-4350-91ca-61724daaee0a`::_resourcesAcquired: Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a (shared) >Thread-120::DEBUG::2014-08-11 15:26:21,975::task::993::Storage.TaskManager.Task::(_decref) Task=`f4c0b0d2-9e6e-4350-91ca-61724daaee0a`::ref 1 aborting False >Thread-14::DEBUG::2014-08-11 15:26:21,976::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:21,976::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:21,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:21,977::__init__::232::IOProcess::(_processLogs) (6994) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:26:21,978::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6994) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:21,979::fileSD::497::Storage.StorageDomain::(activateVolumes) Fixing permissions on /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861 >Thread-14::DEBUG::2014-08-11 15:26:21,980::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:21,980::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-120::DEBUG::2014-08-11 15:26:21,981::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /var/run/vdsm/storage/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:26:21,982::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-120::WARNING::2014-08-11 15:26:21,982::fileUtils::149::Storage.fileUtils::(createdir) Dir /var/run/vdsm/storage/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a already exists >Thread-14::DEBUG::2014-08-11 15:26:21,983::__init__::232::IOProcess::(_processLogs) (6995) Got request for method 'stat' >Thread-120::DEBUG::2014-08-11 15:26:21,983::fileSD::472::Storage.StorageDomain::(createImageLinks) Creating symlink from /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc to /var/run/vdsm/storage/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/1ce118d2-5534-46a2-8322-8d4038f3cadc >Thread-14::DEBUG::2014-08-11 15:26:21,984::__init__::232::IOProcess::(_processLogs) (6995) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:21,984::fileSD::477::Storage.StorageDomain::(createImageLinks) img run dir already exists: /var/run/vdsm/storage/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/1ce118d2-5534-46a2-8322-8d4038f3cadc >Thread-14::DEBUG::2014-08-11 15:26:21,986::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:21,987::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:21,987::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:21,988::__init__::232::IOProcess::(_processLogs) (6996) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:26:21,988::__init__::232::IOProcess::(_processLogs) (6996) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:21,988::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-120::DEBUG::2014-08-11 15:26:21,989::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for d69ba38d-0ec3-4f6e-91a8-12a91d61b861 >Thread-14::DEBUG::2014-08-11 15:26:21,989::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:21,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:21,990::__init__::232::IOProcess::(_processLogs) (6997) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:21,991::__init__::232::IOProcess::(_processLogs) (6997) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:21,991::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:21,992::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:21,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:21,992::__init__::232::IOProcess::(_processLogs) (6998) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:21,993::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6998) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:21,994::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:21,994::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:21,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:21,995::__init__::232::IOProcess::(_processLogs) (6999) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:21,995::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(6999) Queuing response >Thread-120::INFO::2014-08-11 15:26:21,995::logUtils::47::dispatcher::(wrapper) Run and protect: prepareImage, Return response: {'info': {'domainID': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'volumeID': u'd69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'leasePath': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861.lease', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc'}, 'path': u'/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'imgVolumesInfo': [{'domainID': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'volumeID': u'd69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'leasePath': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861.lease', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc'}]} >Thread-120::DEBUG::2014-08-11 15:26:21,996::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4c0b0d2-9e6e-4350-91ca-61724daaee0a`::finished: {'info': {'domainID': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'volumeID': u'd69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'leasePath': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861.lease', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc'}, 'path': u'/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'imgVolumesInfo': [{'domainID': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'volType': 'path', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'volumeID': u'd69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'leasePath': u'/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861.lease', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc'}]} >Thread-120::DEBUG::2014-08-11 15:26:21,997::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4c0b0d2-9e6e-4350-91ca-61724daaee0a`::moving from state preparing -> state finished >Thread-120::DEBUG::2014-08-11 15:26:21,997::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': < ResourceRef 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', isValid: 'True' obj: 'None'>} >Thread-120::DEBUG::2014-08-11 15:26:21,997::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-120::DEBUG::2014-08-11 15:26:21,997::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' >Thread-120::DEBUG::2014-08-11 15:26:21,998::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' (0 active users) >Thread-120::DEBUG::2014-08-11 15:26:21,998::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a' is free, finding out if anyone is waiting for it. >Thread-120::DEBUG::2014-08-11 15:26:21,998::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', Clearing records. >Thread-120::DEBUG::2014-08-11 15:26:21,999::task::993::Storage.TaskManager.Task::(_decref) Task=`f4c0b0d2-9e6e-4350-91ca-61724daaee0a`::ref 0 aborting False >Thread-120::INFO::2014-08-11 15:26:21,999::clientIF::332::vds::(prepareVolumePath) prepared volume path: /rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861 >Thread-115::DEBUG::2014-08-11 15:26:22,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d935851-4adf-4887-b21c-e165cfc35f25`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:22,006::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:26:22,007::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420796', 'lastCheck': '4.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:22,009::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d935851-4adf-4887-b21c-e165cfc35f25`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420796', 'lastCheck': '4.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:22,010::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d935851-4adf-4887-b21c-e165cfc35f25`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:22,011::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:22,012::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:22,013::task::993::Storage.TaskManager.Task::(_decref) Task=`2d935851-4adf-4887-b21c-e165cfc35f25`::ref 0 aborting False >Thread-120::DEBUG::2014-08-11 15:26:22,063::vm::3256::vm.Vm::(_run) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::<?xml version="1.0" encoding="utf-8"?> ><domain type="kvm"> > <name>on-qa</name> > <uuid>e9a797ed-f982-4208-af2a-d6558ca6dfed</uuid> > <memory>1048576</memory> > <currentMemory>1048576</currentMemory> > <vcpu current="1">160</vcpu> > <memtune> > <min_guarantee>1048576</min_guarantee> > </memtune> > <devices> > <channel type="unix"> > <target name="com.redhat.rhevm.vdsm" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/e9a797ed-f982-4208-af2a-d6558ca6dfed.com.redhat.rhevm.vdsm"/> > </channel> > <channel type="unix"> > <target name="org.qemu.guest_agent.0" type="virtio"/> > <source mode="bind" path="/var/lib/libvirt/qemu/channels/e9a797ed-f982-4208-af2a-d6558ca6dfed.org.qemu.guest_agent.0"/> > </channel> > <input bus="ps2" type="mouse"/> > <memballoon model="none"/> > <controller index="0" model="virtio-scsi" type="scsi"> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x04" type="pci"/> > </controller> > <video> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x02" type="pci"/> > <model heads="1" ram="65536" type="qxl" vram="32768"/> > </video> > <graphics autoport="yes" keymap="en-us" passwd="*****" passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice"> > <channel mode="secure" name="main"/> > <channel mode="secure" name="inputs"/> > <channel mode="secure" name="cursor"/> > <channel mode="secure" name="playback"/> > <channel mode="secure" name="record"/> > <channel mode="secure" name="display"/> > <channel mode="secure" name="usbredir"/> > <channel mode="secure" name="smartcard"/> > <listen network="vdsm-ovirtmgmt" type="network"/> > </graphics> > <interface type="bridge"> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x03" type="pci"/> > <mac address="00:1a:4a:16:88:5c"/> > <model type="virtio"/> > <source bridge="ovirtmgmt"/> > <filterref filter="vdsm-no-mac-spoofing"/> > <link state="up"/> > <boot order="1"/> > <driver queues="7"/> > <bandwidth/> > </interface> > <disk device="cdrom" snapshot="no" type="file"> > <address bus="1" controller="0" target="0" type="drive" unit="0"/> > <source file="" startupPolicy="optional"/> > <target bus="ide" dev="hdc"/> > <readonly/> > <serial/> > </disk> > <disk device="disk" snapshot="no" type="file"> > <address bus="0x00" domain="0x0000" function="0x0" slot="0x06" type="pci"/> > <source file="/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861"/> > <target bus="virtio" dev="vda"/> > <serial>1ce118d2-5534-46a2-8322-8d4038f3cadc</serial> > <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> > </disk> > <channel type="spicevmc"> > <target name="com.redhat.spice.0" type="virtio"/> > </channel> > </devices> > <os> > <type arch="x86_64" machine="rhel6.5.0">hvm</type> > <smbios mode="sysinfo"/> > </os> > <sysinfo type="smbios"> > <system> > <entry name="manufacturer">oVirt</entry> > <entry name="product">oVirt Node</entry> > <entry name="version">7.0-1.el7</entry> > <entry name="serial">387D3C42-6B67-3000-8914-EFFAB218B576</entry> > <entry name="uuid">e9a797ed-f982-4208-af2a-d6558ca6dfed</entry> > </system> > </sysinfo> > <clock adjustment="0" offset="variable"> > <timer name="rtc" tickpolicy="catchup"/> > <timer name="pit" tickpolicy="delay"/> > <timer name="hpet" present="no"/> > </clock> > <features> > <acpi/> > </features> > <cpu match="exact"> > <model>Conroe</model> > <topology cores="1" sockets="160" threads="1"/> > </cpu> ></domain> > >libvirtEventLoop::DEBUG::2014-08-11 15:26:22,764::vm::5350::vm.Vm::(_onLibvirtLifecycleEvent) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::event Started detail 0 opaque None >Thread-120::DEBUG::2014-08-11 15:26:22,811::vm::5190::vm.Vm::(_getUnderlyingDriveInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Looking for drive with attributes {'name': u'hdc', 'bootOrder': '', 'boot': [], 'readonly': True, 'address': {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, 'path': '', 'type': u'cdrom'} >Thread-120::DEBUG::2014-08-11 15:26:22,811::vm::5211::vm.Vm::(_getUnderlyingDriveInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Matched {'name': (u'hdc', u'hdc'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, True), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, {u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}), 'path': ('', ''), 'type': (u'cdrom', u'cdrom')} >Thread-120::DEBUG::2014-08-11 15:26:22,812::vm::5230::vm.Vm::(_getUnderlyingDriveInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Matched {'name': (u'hdc', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (True, None), 'address': ({u'bus': u'1', u'controller': u'0', u'type': u'drive', u'target': u'0', u'unit': u'0'}, None), 'path': ('', None), 'type': (u'cdrom', None)} >Thread-120::DEBUG::2014-08-11 15:26:22,812::vm::5190::vm.Vm::(_getUnderlyingDriveInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Looking for drive with attributes {'name': u'vda', 'bootOrder': '', 'boot': [], 'readonly': False, 'address': {u'slot': u'0x06', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, 'path': u'/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', 'type': u'disk'} >Thread-120::DEBUG::2014-08-11 15:26:22,813::vm::5211::vm.Vm::(_getUnderlyingDriveInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Matched {'name': (u'vda', u'vda'), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (False, False), 'address': ({u'slot': u'0x06', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, {u'slot': u'0x06', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}), 'path': (u'/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', u'/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861'), 'type': (u'disk', u'disk')} >Thread-120::DEBUG::2014-08-11 15:26:22,813::vm::5230::vm.Vm::(_getUnderlyingDriveInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Matched {'name': (u'vda', None), 'bootOrder': ('', None), 'boot': ([], None), 'readonly': (False, None), 'address': ({u'slot': u'0x06', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'function': u'0x0'}, None), 'path': (u'/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/images/1ce118d2-5534-46a2-8322-8d4038f3cadc/d69ba38d-0ec3-4f6e-91a8-12a91d61b861', None), 'type': (u'disk', None)} >Thread-120::DEBUG::2014-08-11 15:26:22,946::vm::5666::vm.Vm::(_syncVolumeChain) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::vdsm chain: [u'd69ba38d-0ec3-4f6e-91a8-12a91d61b861'], libvirt chain: [u'd69ba38d-0ec3-4f6e-91a8-12a91d61b861'] >Thread-120::DEBUG::2014-08-11 15:26:22,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`945c9700-7974-4d5c-ae95-e7081bf19654`::moving from state init -> state preparing >Thread-120::INFO::2014-08-11 15:26:22,947::logUtils::44::dispatcher::(wrapper) Run and protect: imageSyncVolumeChain(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', newChain=[u'd69ba38d-0ec3-4f6e-91a8-12a91d61b861']) >Thread-120::DEBUG::2014-08-11 15:26:22,948::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc`ReqID=`a85e54d8-1b1c-46c2-afa6-15c660a55c8c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1821' at 'imageSyncVolumeChain' >Thread-120::DEBUG::2014-08-11 15:26:22,948::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc' for lock type 'shared' >Thread-14::DEBUG::2014-08-11 15:26:22,949::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,949::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,950::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,950::__init__::232::IOProcess::(_processLogs) (7000) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:26:22,950::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7000) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,951::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,952::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,952::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,952::__init__::232::IOProcess::(_processLogs) (7001) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:26:22,952::__init__::232::IOProcess::(_processLogs) (7001) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,953::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,953::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,953::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,954::__init__::232::IOProcess::(_processLogs) (7002) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,954::__init__::232::IOProcess::(_processLogs) (7002) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:22,954::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for d69ba38d-0ec3-4f6e-91a8-12a91d61b861 >Thread-14::DEBUG::2014-08-11 15:26:22,955::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,955::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,955::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,955::__init__::232::IOProcess::(_processLogs) (7003) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,956::__init__::232::IOProcess::(_processLogs) (7003) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,956::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,957::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,957::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,957::__init__::232::IOProcess::(_processLogs) (7004) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,957::__init__::232::IOProcess::(_processLogs) (7004) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,958::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,958::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,959::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,959::__init__::232::IOProcess::(_processLogs) (7005) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:26:22,959::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7005) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,960::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,960::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,961::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,961::__init__::232::IOProcess::(_processLogs) (7006) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:26:22,961::__init__::232::IOProcess::(_processLogs) (7006) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,962::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,962::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,963::__init__::232::IOProcess::(_processLogs) (7007) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,963::__init__::232::IOProcess::(_processLogs) (7007) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:22,963::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for d69ba38d-0ec3-4f6e-91a8-12a91d61b861 >Thread-14::DEBUG::2014-08-11 15:26:22,963::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,964::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,964::__init__::232::IOProcess::(_processLogs) (7008) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,965::__init__::232::IOProcess::(_processLogs) (7008) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,965::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,966::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,966::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,966::__init__::232::IOProcess::(_processLogs) (7009) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,966::__init__::232::IOProcess::(_processLogs) (7009) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,967::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,967::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,968::__init__::232::IOProcess::(_processLogs) (7010) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:26:22,968::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7010) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,969::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,969::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,969::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,970::__init__::232::IOProcess::(_processLogs) (7011) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:26:22,970::__init__::232::IOProcess::(_processLogs) (7011) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,970::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,971::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,971::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,971::__init__::232::IOProcess::(_processLogs) (7012) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,971::__init__::232::IOProcess::(_processLogs) (7012) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:22,972::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for d69ba38d-0ec3-4f6e-91a8-12a91d61b861 >Thread-14::DEBUG::2014-08-11 15:26:22,972::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,972::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,973::__init__::232::IOProcess::(_processLogs) (7013) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,973::__init__::232::IOProcess::(_processLogs) (7013) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,974::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,974::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,974::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,975::__init__::232::IOProcess::(_processLogs) (7014) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,975::__init__::232::IOProcess::(_processLogs) (7014) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,975::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,976::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,976::__init__::232::IOProcess::(_processLogs) (7015) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:26:22,977::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7015) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,978::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,978::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,978::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,978::__init__::232::IOProcess::(_processLogs) (7016) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:26:22,979::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7016) Queuing response >Thread-120::INFO::2014-08-11 15:26:22,979::image::213::Storage.Image::(getChain) sdUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a imgUUID=1ce118d2-5534-46a2-8322-8d4038f3cadc chain=[<storage.fileVolume.FileVolume object at 0x240d250>] >Thread-14::DEBUG::2014-08-11 15:26:22,980::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,980::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,981::__init__::232::IOProcess::(_processLogs) (7017) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:26:22,981::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7017) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:22,982::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861`ReqID=`47f9e8c2-0556-48a5-94e5-177514b5cb9b`::Request was made in '/usr/share/vdsm/storage/resourceFactories.py' line '172' at '__getResourceCandidatesList' >Thread-120::DEBUG::2014-08-11 15:26:22,982::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861' for lock type 'shared' >Thread-120::DEBUG::2014-08-11 15:26:22,982::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861' is free. Now locking as 'shared' (1 active user) >Thread-120::DEBUG::2014-08-11 15:26:22,982::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861`ReqID=`47f9e8c2-0556-48a5-94e5-177514b5cb9b`::Granted request >Thread-120::DEBUG::2014-08-11 15:26:22,983::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc' is free. Now locking as 'shared' (1 active user) >Thread-120::DEBUG::2014-08-11 15:26:22,983::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc`ReqID=`a85e54d8-1b1c-46c2-afa6-15c660a55c8c`::Granted request >Thread-14::DEBUG::2014-08-11 15:26:22,984::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,985::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,985::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,985::__init__::232::IOProcess::(_processLogs) (7018) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:26:22,985::__init__::232::IOProcess::(_processLogs) (7018) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,986::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,986::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,987::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,987::__init__::232::IOProcess::(_processLogs) (7019) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,987::__init__::232::IOProcess::(_processLogs) (7019) Queuing response >Thread-120::DEBUG::2014-08-11 15:26:22,987::fileVolume::529::Storage.Volume::(validateVolumePath) validate path for d69ba38d-0ec3-4f6e-91a8-12a91d61b861 >Thread-14::DEBUG::2014-08-11 15:26:22,988::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,988::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,988::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,989::__init__::232::IOProcess::(_processLogs) (7020) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,989::__init__::232::IOProcess::(_processLogs) (7020) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,989::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,990::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,990::__init__::232::IOProcess::(_processLogs) (7021) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:22,990::__init__::232::IOProcess::(_processLogs) (7021) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,991::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,991::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,992::__init__::232::IOProcess::(_processLogs) (7022) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:26:22,992::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7022) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:22,993::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:22,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:22,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:22,994::__init__::232::IOProcess::(_processLogs) (7023) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:26:22,995::__init__::232::IOProcess::(_processLogs) (7023) Queuing response >Thread-120::INFO::2014-08-11 15:26:22,995::image::213::Storage.Image::(getChain) sdUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a imgUUID=1ce118d2-5534-46a2-8322-8d4038f3cadc chain=[<storage.fileVolume.FileVolume object at 0x7f71f0761190>] >Thread-120::DEBUG::2014-08-11 15:26:22,995::hsm::1829::Storage.HSM::(imageSyncVolumeChain) unlinking subchain: [] >Thread-120::DEBUG::2014-08-11 15:26:22,996::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc' >Thread-120::DEBUG::2014-08-11 15:26:22,996::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc' (0 active users) >Thread-120::DEBUG::2014-08-11 15:26:22,996::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc' is free, finding out if anyone is waiting for it. >Thread-120::DEBUG::2014-08-11 15:26:22,996::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861' >Thread-120::DEBUG::2014-08-11 15:26:22,997::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861' (0 active users) >Thread-120::DEBUG::2014-08-11 15:26:22,997::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861' is free, finding out if anyone is waiting for it. >Thread-120::DEBUG::2014-08-11 15:26:22,997::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS.d69ba38d-0ec3-4f6e-91a8-12a91d61b861', Clearing records. >Thread-120::DEBUG::2014-08-11 15:26:22,997::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS.1ce118d2-5534-46a2-8322-8d4038f3cadc', Clearing records. >Thread-120::INFO::2014-08-11 15:26:22,998::logUtils::47::dispatcher::(wrapper) Run and protect: imageSyncVolumeChain, Return response: None >Thread-120::DEBUG::2014-08-11 15:26:22,998::task::1191::Storage.TaskManager.Task::(prepare) Task=`945c9700-7974-4d5c-ae95-e7081bf19654`::finished: None >Thread-120::DEBUG::2014-08-11 15:26:22,998::task::595::Storage.TaskManager.Task::(_updateState) Task=`945c9700-7974-4d5c-ae95-e7081bf19654`::moving from state preparing -> state finished >Thread-120::DEBUG::2014-08-11 15:26:22,998::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-120::DEBUG::2014-08-11 15:26:22,999::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-120::DEBUG::2014-08-11 15:26:22,999::task::993::Storage.TaskManager.Task::(_decref) Task=`945c9700-7974-4d5c-ae95-e7081bf19654`::ref 0 aborting False >Thread-120::DEBUG::2014-08-11 15:26:23,000::sampling::399::vm.Vm::(start) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Start statistics collection >Thread-122::DEBUG::2014-08-11 15:26:23,000::sampling::428::vm.Vm::(run) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Stats thread started >Thread-122::DEBUG::2014-08-11 15:26:23,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`ded08a4d-a455-4b7a-b2e7-ac23eea125f9`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:26:23,003::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-120::DEBUG::2014-08-11 15:26:23,004::vmchannels::199::vds::(register) Add fileno 14 to listener's channels. >Thread-14::DEBUG::2014-08-11 15:26:23,005::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:23,006::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:23,006::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:23,008::__init__::232::IOProcess::(_processLogs) (7024) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:26:23,008::__init__::232::IOProcess::(_processLogs) (7024) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:23,008::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:23,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:26:23,009::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:26:23,010::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:23,011::__init__::232::IOProcess::(_processLogs) (7025) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:26:23,011::task::1191::Storage.TaskManager.Task::(prepare) Task=`ded08a4d-a455-4b7a-b2e7-ac23eea125f9`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:26:23,013::__init__::232::IOProcess::(_processLogs) (7025) Queuing response >Thread-122::DEBUG::2014-08-11 15:26:23,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`ded08a4d-a455-4b7a-b2e7-ac23eea125f9`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:26:23,014::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:26:23,014::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:26:23,015::task::993::Storage.TaskManager.Task::(_decref) Task=`ded08a4d-a455-4b7a-b2e7-ac23eea125f9`::ref 0 aborting False >Thread-120::WARNING::2014-08-11 15:26:23,029::vm::3893::vm.Vm::(_readPauseCode) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::_readPauseCode unsupported by libvirt vm >Thread-120::DEBUG::2014-08-11 15:26:23,054::vm::2254::vm.Vm::(_startUnderlyingVm) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::_ongoingCreations released >Dummy-33::DEBUG::2014-08-11 15:26:23,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:23,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013104 s, 78.1 MB/s\n'; <rc> = 0 >VM Channels Listener::DEBUG::2014-08-11 15:26:23,439::vmchannels::107::vds::(_do_add_channels) fileno 14 was added to unconnected channels. >VM Channels Listener::DEBUG::2014-08-11 15:26:23,440::vmchannels::133::vds::(_handle_unconnected) Trying to connect fileno 14. >VM Channels Listener::DEBUG::2014-08-11 15:26:23,440::guestagent::199::vm.Vm::(_connect) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Attempting connection to /var/lib/libvirt/qemu/channels/e9a797ed-f982-4208-af2a-d6558ca6dfed.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-08-11 15:26:23,441::guestagent::202::vm.Vm::(_connect) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Connected to /var/lib/libvirt/qemu/channels/e9a797ed-f982-4208-af2a-d6558ca6dfed.com.redhat.rhevm.vdsm >VM Channels Listener::DEBUG::2014-08-11 15:26:23,441::vmchannels::149::vds::(_handle_unconnected) Connecting to fileno 14 succeeded. >Thread-115::DEBUG::2014-08-11 15:26:25,270::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:25,271::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'hash': '5052350335364909575', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'displayIp': '10.35.128.39', 'displayPort': '-1', 'session': 'Unknown', 'displaySecurePort': u'5900', 'timeOffset': '0', 'memUsage': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '0', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'elapsedTime': '3', 'kvmEnable': 'true', 'network': {}, 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'vmType': 'kvm', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.13', 'username': 'Unknown', 'displayType': 'qxl', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >Dummy-33::DEBUG::2014-08-11 15:26:25,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:25,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132223 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:27,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:27,340::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:27,438::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:26:27,439::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:26:27,439::__init__::232::IOProcess::(_processLogs) (7026) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:27,457::__init__::232::IOProcess::(_processLogs) (7026) Queuing response >Thread-16::DEBUG::2014-08-11 15:26:27,459::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000475932 s, 956 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:27,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:27,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:27,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:27,461::__init__::232::IOProcess::(_processLogs) (7027) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:27,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7027) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:27,464::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Dummy-33::DEBUG::2014-08-11 15:26:27,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:26:27,465::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:27,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:27,482::__init__::232::IOProcess::(_processLogs) (7028) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:27,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7028) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:27,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:27,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:27,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:27,485::__init__::232::IOProcess::(_processLogs) (7029) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:27,485::__init__::232::IOProcess::(_processLogs) (7029) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:27,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:27,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:27,486::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:27,487::__init__::232::IOProcess::(_processLogs) (7030) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:27,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7030) Queuing response >Dummy-33::DEBUG::2014-08-11 15:26:27,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141289 s, 72.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:28,339::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:28,340::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'hash': '5052350335364909575', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'displayIp': '10.35.128.39', 'displayPort': '-1', 'session': 'Unknown', 'displaySecurePort': u'5900', 'timeOffset': '0', 'memUsage': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '0', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'elapsedTime': '6', 'kvmEnable': 'true', 'network': {}, 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'vmType': 'kvm', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.19', 'username': 'Unknown', 'displayType': 'qxl', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >Dummy-33::DEBUG::2014-08-11 15:26:29,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:29,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134554 s, 76.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:29,925::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:26:29,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`665c6435-e2d5-4f7f-b497-02772bcfc226`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:29,926::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:26:29,956::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:29,957::task::1191::Storage.TaskManager.Task::(prepare) Task=`665c6435-e2d5-4f7f-b497-02772bcfc226`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:29,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`665c6435-e2d5-4f7f-b497-02772bcfc226`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:29,957::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:29,958::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:29,958::task::993::Storage.TaskManager.Task::(_decref) Task=`665c6435-e2d5-4f7f-b497-02772bcfc226`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:26:29,974::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:26:29,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6fa6415-99a1-4555-ac5f-e5ad1b0310c0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:26:29,975::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:26:29,976::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`02f55434-185a-4022-aa60-1792ed60c2f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:26:29,976::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:26:29,976::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:26:29,976::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`02f55434-185a-4022-aa60-1792ed60c2f9`::Granted request >Thread-34::DEBUG::2014-08-11 15:26:29,977::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6fa6415-99a1-4555-ac5f-e5ad1b0310c0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:26:29,977::task::993::Storage.TaskManager.Task::(_decref) Task=`b6fa6415-99a1-4555-ac5f-e5ad1b0310c0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:26:30,011::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:30,012::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6fa6415-99a1-4555-ac5f-e5ad1b0310c0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:30,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6fa6415-99a1-4555-ac5f-e5ad1b0310c0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:26:30,012::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:26:30,012::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:26:30,013::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:26:30,013::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:26:30,013::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:26:30,013::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:26:30,014::task::993::Storage.TaskManager.Task::(_decref) Task=`b6fa6415-99a1-4555-ac5f-e5ad1b0310c0`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:26:31,362::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:31,363::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'hash': '5052350335364909575', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'displayIp': '10.35.128.39', 'displayPort': '-1', 'session': 'Unknown', 'displaySecurePort': u'5900', 'timeOffset': '0', 'memUsage': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '0', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'elapsedTime': '9', 'kvmEnable': 'true', 'network': {}, 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'vmType': 'kvm', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.22', 'username': 'Unknown', 'displayType': 'qxl', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >Dummy-33::DEBUG::2014-08-11 15:26:31,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:31,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132047 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:26:33,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:33,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133255 s, 76.8 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:34,390::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:34,391::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'status': 'Powering up', 'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'hash': '5052350335364909575', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'displayIp': '10.35.128.39', 'displayPort': '-1', 'session': 'Unknown', 'displaySecurePort': u'5900', 'timeOffset': '0', 'memUsage': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '0', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'clientIp': '', 'elapsedTime': '12', 'kvmEnable': 'true', 'network': {}, 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'vmType': 'kvm', 'cpuUser': '0.00', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.24', 'username': 'Unknown', 'displayType': 'qxl', 'cpuSys': '0.00', 'guestCPUCount': -1, 'appsList': [], 'guestIPs': ''}]} >Dummy-33::DEBUG::2014-08-11 15:26:35,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:35,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134133 s, 76.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:37,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`89dcc00e-0966-4f3a-83a2-608d8715caa9`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:37,408::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:26:37,409::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000475932', 'lastCheck': '9.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:37,409::task::1191::Storage.TaskManager.Task::(prepare) Task=`89dcc00e-0966-4f3a-83a2-608d8715caa9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000475932', 'lastCheck': '9.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:37,409::task::595::Storage.TaskManager.Task::(_updateState) Task=`89dcc00e-0966-4f3a-83a2-608d8715caa9`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:37,409::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:37,410::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:37,410::task::993::Storage.TaskManager.Task::(_decref) Task=`89dcc00e-0966-4f3a-83a2-608d8715caa9`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:26:37,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:26:37,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:37,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:37,491::__init__::232::IOProcess::(_processLogs) (7031) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:26:37,491::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:26:37,516::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00594594 s, 76.5 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:37,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:37,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:37,518::__init__::232::IOProcess::(_processLogs) (7032) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:37,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:37,520::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:37,520::__init__::232::IOProcess::(_processLogs) (7033) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:37,520::__init__::232::IOProcess::(_processLogs) (7033) Queuing response >Thread-14::DEBUG::2014-08-11 15:26:37,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:37,521::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:37,521::__init__::232::IOProcess::(_processLogs) (7034) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:37,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:37,523::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:37,523::__init__::232::IOProcess::(_processLogs) (7035) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:37,523::__init__::232::IOProcess::(_processLogs) (7035) Queuing response >Dummy-33::DEBUG::2014-08-11 15:26:37,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:37,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0413904 s, 24.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:26:39,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:39,926::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133211 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:40,038::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-115::DEBUG::2014-08-11 15:26:40,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`2fa9be5a-2e61-4417-8274-623d09a1a79d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:40,040::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:26:40,052::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:40,053::task::1191::Storage.TaskManager.Task::(prepare) Task=`2fa9be5a-2e61-4417-8274-623d09a1a79d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:40,053::task::595::Storage.TaskManager.Task::(_updateState) Task=`2fa9be5a-2e61-4417-8274-623d09a1a79d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:40,053::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:40,053::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:40,054::task::993::Storage.TaskManager.Task::(_decref) Task=`2fa9be5a-2e61-4417-8274-623d09a1a79d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:26:40,064::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 15:26:40,065::task::595::Storage.TaskManager.Task::(_updateState) Task=`029e7f5b-bc5f-4123-b55c-f7171844a2cd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:26:40,065::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:26:40,066::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1aee05af-d66a-4da8-99b7-66eab07ded9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:26:40,066::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:26:40,066::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:26:40,066::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1aee05af-d66a-4da8-99b7-66eab07ded9e`::Granted request >Thread-34::DEBUG::2014-08-11 15:26:40,067::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`029e7f5b-bc5f-4123-b55c-f7171844a2cd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:26:40,067::task::993::Storage.TaskManager.Task::(_decref) Task=`029e7f5b-bc5f-4123-b55c-f7171844a2cd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:26:40,087::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:40,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`029e7f5b-bc5f-4123-b55c-f7171844a2cd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959424512', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:40,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`029e7f5b-bc5f-4123-b55c-f7171844a2cd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:26:40,088::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:26:40,088::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:26:40,088::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:26:40,089::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:26:40,089::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:26:40,089::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:26:40,090::task::993::Storage.TaskManager.Task::(_decref) Task=`029e7f5b-bc5f-4123-b55c-f7171844a2cd`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:26:40,469::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:40,470::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:26:40,471::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '278', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.15', 'elapsedTime': '18', 'displayType': 'qxl', 'cpuSys': '56.03', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.21', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:26:41,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:43,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.44236 s, 710 kB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:43,493::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:43,494::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:26:43,495::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '278', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.15', 'elapsedTime': '21', 'displayType': 'qxl', 'cpuSys': '56.03', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.24', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:26:45,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:45,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133494 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:45,618::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:45,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:45,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:26:46,523::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:46,524::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:26:46,524::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '278', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.15', 'elapsedTime': '24', 'displayType': 'qxl', 'cpuSys': '56.03', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.27', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:26:47,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:26:47,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:47,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:47,527::__init__::232::IOProcess::(_processLogs) (7036) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:26:47,528::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:26:47,549::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000600284 s, 758 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:47,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 15:26:47,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0378984 s, 27.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:47,555::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:47,557::__init__::232::IOProcess::(_processLogs) (7037) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:47,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:47,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:47,563::__init__::232::IOProcess::(_processLogs) (7038) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:47,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:47,568::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:47,569::__init__::232::IOProcess::(_processLogs) (7039) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:47,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:26:47,573::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:26:47,574::__init__::232::IOProcess::(_processLogs) (7040) Got request for method 'access' >Thread-115::DEBUG::2014-08-11 15:26:49,546::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:49,547::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:26:49,547::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '278', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.15', 'elapsedTime': '27', 'displayType': 'qxl', 'cpuSys': '56.03', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.29', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:26:49,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:49,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132102 s, 77.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:50,102::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:26:50,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`67b9b89f-ec8f-49a9-96b3-030d03c77713`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:50,104::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:26:50,116::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:50,117::task::1191::Storage.TaskManager.Task::(prepare) Task=`67b9b89f-ec8f-49a9-96b3-030d03c77713`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:26:50,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`67b9b89f-ec8f-49a9-96b3-030d03c77713`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:50,117::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:50,117::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:50,118::task::993::Storage.TaskManager.Task::(_decref) Task=`67b9b89f-ec8f-49a9-96b3-030d03c77713`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:26:50,128::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:26:50,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec3885f8-380c-4f7e-b706-64d9a12be88d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:26:50,129::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:26:50,129::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`268076a6-4cdc-428d-98a0-ccb75fbc5fbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:26:50,130::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:26:50,130::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:26:50,130::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`268076a6-4cdc-428d-98a0-ccb75fbc5fbc`::Granted request >Thread-34::DEBUG::2014-08-11 15:26:50,131::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ec3885f8-380c-4f7e-b706-64d9a12be88d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:26:50,131::task::993::Storage.TaskManager.Task::(_decref) Task=`ec3885f8-380c-4f7e-b706-64d9a12be88d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:26:50,151::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:50,151::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec3885f8-380c-4f7e-b706-64d9a12be88d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:26:50,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec3885f8-380c-4f7e-b706-64d9a12be88d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:26:50,152::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:26:50,152::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:26:50,152::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:26:50,153::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:26:50,153::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:26:50,153::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:26:50,153::task::993::Storage.TaskManager.Task::(_decref) Task=`ec3885f8-380c-4f7e-b706-64d9a12be88d`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:50,622::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:50,625::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:50,626::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:26:51,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:51,683::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133212 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:52,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`51950f97-2de8-470f-9aa5-6b013dfdd64e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:26:52,565::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:26:52,565::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000600284', 'lastCheck': '5.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:52,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`51950f97-2de8-470f-9aa5-6b013dfdd64e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000600284', 'lastCheck': '5.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:26:52,566::task::595::Storage.TaskManager.Task::(_updateState) Task=`51950f97-2de8-470f-9aa5-6b013dfdd64e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:26:52,566::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:26:52,566::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:26:52,566::task::993::Storage.TaskManager.Task::(_decref) Task=`51950f97-2de8-470f-9aa5-6b013dfdd64e`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:26:52,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >VM Channels Listener::DEBUG::2014-08-11 15:26:53,498::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 14. >Dummy-33::DEBUG::2014-08-11 15:26:53,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:53,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184531 s, 55.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:55,622::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:55,623::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:26:55,624::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '566', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.54', 'elapsedTime': '33', 'displayType': 'qxl', 'cpuSys': '54.53', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.35', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:55,632::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:55,633::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:26:55,634::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:26:55,766::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:55,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133035 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:57,577::__init__::232::IOProcess::(_processLogs) (7041) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:26:57,578::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:26:57,596::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415581 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:26:57,598::__init__::232::IOProcess::(_processLogs) (7042) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:26:57,599::__init__::232::IOProcess::(_processLogs) (7043) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:57,600::__init__::232::IOProcess::(_processLogs) (7044) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:26:57,601::__init__::232::IOProcess::(_processLogs) (7045) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:26:57,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:57,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132554 s, 77.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:26:58,645::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:26:58,646::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:26:58,646::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '566', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.54', 'elapsedTime': '36', 'displayType': 'qxl', 'cpuSys': '54.53', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.37', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:26:59,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:26:59,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136304 s, 75.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:00,164::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:27:00,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`1890bb34-1133-48f2-8282-a41f548aa6d4`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:00,166::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:27:00,179::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:00,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`1890bb34-1133-48f2-8282-a41f548aa6d4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:00,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`1890bb34-1133-48f2-8282-a41f548aa6d4`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:00,180::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:00,180::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:00,180::task::993::Storage.TaskManager.Task::(_decref) Task=`1890bb34-1133-48f2-8282-a41f548aa6d4`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:27:00,191::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:27:00,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbca3eff-436e-4691-ab18-c3ed5ed75499`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:27:00,192::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:27:00,193::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c2b5c982-de14-4674-b3c9-cace7e7edb52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:27:00,193::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:27:00,193::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:27:00,194::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c2b5c982-de14-4674-b3c9-cace7e7edb52`::Granted request >Thread-34::DEBUG::2014-08-11 15:27:00,194::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cbca3eff-436e-4691-ab18-c3ed5ed75499`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:27:00,194::task::993::Storage.TaskManager.Task::(_decref) Task=`cbca3eff-436e-4691-ab18-c3ed5ed75499`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:27:00,217::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:00,217::task::1191::Storage.TaskManager.Task::(prepare) Task=`cbca3eff-436e-4691-ab18-c3ed5ed75499`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:00,218::task::595::Storage.TaskManager.Task::(_updateState) Task=`cbca3eff-436e-4691-ab18-c3ed5ed75499`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:27:00,218::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:27:00,218::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:27:00,218::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:27:00,219::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:27:00,219::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:27:00,219::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:27:00,219::task::993::Storage.TaskManager.Task::(_decref) Task=`cbca3eff-436e-4691-ab18-c3ed5ed75499`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:00,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:00,641::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:00,642::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:01,667::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:27:01,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:01,669::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '566', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.54', 'elapsedTime': '39', 'displayType': 'qxl', 'cpuSys': '54.53', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.40', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:27:01,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:02,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0303067 s, 33.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:27:04,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:04,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133189 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:04,691::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} flowID [4e931c41] >Thread-115::DEBUG::2014-08-11 15:27:04,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:04,692::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '566', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '16.54', 'elapsedTime': '42', 'displayType': 'qxl', 'cpuSys': '54.53', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.42', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:05,648::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:05,649::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:05,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:06,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:06,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135026 s, 75.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:07,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:07,604::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:07,604::__init__::232::IOProcess::(_processLogs) (7046) Queuing response >Thread-16::DEBUG::2014-08-11 15:27:07,605::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:27:07,623::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000519297 s, 876 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:07,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:07,625::__init__::232::IOProcess::(_processLogs) (7047) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:27:07,626::__init__::232::IOProcess::(_processLogs) (7048) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:07,627::__init__::232::IOProcess::(_processLogs) (7048) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:07,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:07,628::__init__::232::IOProcess::(_processLogs) (7049) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:07,628::__init__::232::IOProcess::(_processLogs) (7049) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:07,629::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:07,629::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:07,630::__init__::232::IOProcess::(_processLogs) (7050) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:07,630::__init__::232::IOProcess::(_processLogs) (7050) Queuing response >Thread-115::DEBUG::2014-08-11 15:27:07,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`e20c8712-b077-4524-a841-1aeb6fb8c02f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:07,711::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:27:07,712::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000519297', 'lastCheck': '0.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:07,712::task::1191::Storage.TaskManager.Task::(prepare) Task=`e20c8712-b077-4524-a841-1aeb6fb8c02f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000519297', 'lastCheck': '0.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:07,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`e20c8712-b077-4524-a841-1aeb6fb8c02f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:07,712::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:07,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:07,713::task::993::Storage.TaskManager.Task::(_decref) Task=`e20c8712-b077-4524-a841-1aeb6fb8c02f`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:27:07,734::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:08,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:08,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135156 s, 75.8 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:10,232::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:27:10,234::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0f06d13-a9e3-414c-84ca-c2cbbd96cd32`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:10,234::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:27:10,253::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:10,253::task::1191::Storage.TaskManager.Task::(prepare) Task=`a0f06d13-a9e3-414c-84ca-c2cbbd96cd32`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:10,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`a0f06d13-a9e3-414c-84ca-c2cbbd96cd32`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:10,254::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:10,254::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:10,254::task::993::Storage.TaskManager.Task::(_decref) Task=`a0f06d13-a9e3-414c-84ca-c2cbbd96cd32`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:27:10,266::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:27:10,266::task::595::Storage.TaskManager.Task::(_updateState) Task=`25a32f75-44a5-4b78-bcc4-28977be2d4e0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:27:10,267::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:27:10,268::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a43efb4-3bfa-4974-96ec-245721e589af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:27:10,268::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:27:10,270::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:27:10,270::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a43efb4-3bfa-4974-96ec-245721e589af`::Granted request >Thread-34::DEBUG::2014-08-11 15:27:10,271::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`25a32f75-44a5-4b78-bcc4-28977be2d4e0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:27:10,271::task::993::Storage.TaskManager.Task::(_decref) Task=`25a32f75-44a5-4b78-bcc4-28977be2d4e0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:27:10,291::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:10,292::task::1191::Storage.TaskManager.Task::(prepare) Task=`25a32f75-44a5-4b78-bcc4-28977be2d4e0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:10,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`25a32f75-44a5-4b78-bcc4-28977be2d4e0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:27:10,292::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:27:10,292::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:27:10,293::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:27:10,293::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:27:10,293::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:27:10,293::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:27:10,294::task::993::Storage.TaskManager.Task::(_decref) Task=`25a32f75-44a5-4b78-bcc4-28977be2d4e0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:27:10,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:10,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135395 s, 75.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:10,657::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:10,658::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:10,659::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:10,778::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} flowID [5480b349] >Thread-115::DEBUG::2014-08-11 15:27:10,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:10,780::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '902', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '15.13', 'elapsedTime': '48', 'displayType': 'qxl', 'cpuSys': '43.61', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.48', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:27:12,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:12,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131475 s, 77.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:13,812::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:27:13,815::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:13,817::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '902', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '15.13', 'elapsedTime': '51', 'displayType': 'qxl', 'cpuSys': '43.61', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.52', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Dummy-33::DEBUG::2014-08-11 15:27:14,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:14,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219082 s, 46.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:15,666::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:15,667::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:15,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:16,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:16,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135915 s, 75.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:16,838::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:27:16,840::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:16,840::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '902', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '15.13', 'elapsedTime': '54', 'displayType': 'qxl', 'cpuSys': '43.61', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '1.54', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Thread-14::DEBUG::2014-08-11 15:27:17,632::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:17,632::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:17,640::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:27:17,641::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:27:17,660::__init__::232::IOProcess::(_processLogs) (7051) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:27:17,663::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000822712 s, 553 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:17,662::__init__::232::IOProcess::(_processLogs) (7051) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:17,664::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:17,664::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:17,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:17,665::__init__::232::IOProcess::(_processLogs) (7052) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:27:17,665::__init__::232::IOProcess::(_processLogs) DEBUG|(7052) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:17,666::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:17,667::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:17,668::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:17,668::__init__::232::IOProcess::(_processLogs) (7053) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:17,668::__init__::232::IOProcess::(_processLogs) (7053) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:17,669::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:17,669::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:17,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:17,671::__init__::232::IOProcess::(_processLogs) (7054) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:17,671::__init__::232::IOProcess::(_processLogs) (7054) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:17,671::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:17,672::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:17,672::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:17,673::__init__::232::IOProcess::(_processLogs) (7055) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:17,674::__init__::232::IOProcess::(_processLogs) (7055) Queuing response >Dummy-33::DEBUG::2014-08-11 15:27:18,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:18,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134501 s, 76.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:19,863::BindingXMLRPC::1127::vds::(wrapper) client [10.35.162.23]::call vmGetStats with ('e9a797ed-f982-4208-af2a-d6558ca6dfed',) {} >Thread-115::DEBUG::2014-08-11 15:27:19,864::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:27:19,865::BindingXMLRPC::1134::vds::(wrapper) return vmGetStats with {'status': {'message': 'Done', 'code': 0}, 'statsList': [{'displayInfo': [{'tlsPort': u'5900', 'ipAddress': '10.35.128.39', 'type': 'spice', 'port': '-1'}], 'memUsage': '0', 'acpiEnable': 'true', 'guestFQDN': '', 'pid': '707', 'session': 'Unknown', 'vmType': 'kvm', 'timeOffset': '0', 'balloonInfo': {'balloon_max': '1048576', 'balloon_target': '0', 'balloon_cur': '1048576', 'balloon_min': '1048576'}, 'pauseCode': 'NOERR', 'network': {u'vnet0': {'macAddr': '00:1a:4a:16:88:5c', 'rxDropped': '902', 'rxErrors': '0', 'txDropped': '0', 'txRate': '0.0', 'rxRate': '0.0', 'txErrors': '0', 'state': 'unknown', 'speed': '1000', 'name': u'vnet0'}}, 'vmJobs': {}, 'cpuUser': '15.13', 'elapsedTime': '57', 'displayType': 'qxl', 'cpuSys': '43.61', 'appsList': [], 'vcpuCount': '1', 'clientIp': '', 'hash': '5052350335364909575', 'vmId': 'e9a797ed-f982-4208-af2a-d6558ca6dfed', 'displayIp': '10.35.128.39', 'vcpuPeriod': 100000L, 'displayPort': '-1', 'vcpuQuota': '-1', 'kvmEnable': 'true', 'disks': {u'vda': {'readLatency': '0', 'apparentsize': '10737418240', 'writeLatency': '0', 'imageID': '1ce118d2-5534-46a2-8322-8d4038f3cadc', 'flushLatency': '0', 'truesize': '1169743872'}, u'hdc': {'flushLatency': '0', 'readLatency': '0', 'truesize': '0', 'apparentsize': '0', 'writeLatency': '0'}}, 'monitorResponse': '0', 'statsAge': '0.57', 'username': 'Unknown', 'status': 'Powering up', 'guestCPUCount': -1, 'guestIPs': '', 'displaySecurePort': u'5900'}]} >Thread-115::DEBUG::2014-08-11 15:27:20,305::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-115::DEBUG::2014-08-11 15:27:20,306::task::595::Storage.TaskManager.Task::(_updateState) Task=`057f720d-e399-4c83-b669-5d30ecfc1465`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:20,307::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:27:20,327::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:20,327::task::1191::Storage.TaskManager.Task::(prepare) Task=`057f720d-e399-4c83-b669-5d30ecfc1465`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:20,328::task::595::Storage.TaskManager.Task::(_updateState) Task=`057f720d-e399-4c83-b669-5d30ecfc1465`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:20,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:20,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:20,328::task::993::Storage.TaskManager.Task::(_decref) Task=`057f720d-e399-4c83-b669-5d30ecfc1465`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:27:20,339::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 15:27:20,340::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdeaf548-f52c-4b2d-9f1e-f4424fd5321f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:27:20,341::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:27:20,341::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aad87233-d411-4dd9-bec3-350e72220ee0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:27:20,342::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:27:20,342::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:27:20,342::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`aad87233-d411-4dd9-bec3-350e72220ee0`::Granted request >Thread-34::DEBUG::2014-08-11 15:27:20,343::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bdeaf548-f52c-4b2d-9f1e-f4424fd5321f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:27:20,343::task::993::Storage.TaskManager.Task::(_decref) Task=`bdeaf548-f52c-4b2d-9f1e-f4424fd5321f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:27:20,364::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:20,365::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdeaf548-f52c-4b2d-9f1e-f4424fd5321f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:20,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdeaf548-f52c-4b2d-9f1e-f4424fd5321f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:27:20,365::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:27:20,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:27:20,366::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:27:20,366::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:27:20,367::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:27:20,367::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:27:20,367::task::993::Storage.TaskManager.Task::(_decref) Task=`bdeaf548-f52c-4b2d-9f1e-f4424fd5321f`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:20,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:20,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:20,678::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:20,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:20,846::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134521 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:27:22,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-115::DEBUG::2014-08-11 15:27:22,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`aec435e2-670f-4480-8640-6e6dbed75dd9`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:22,890::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:27:22,891::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000822712', 'lastCheck': '5.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:22,891::task::1191::Storage.TaskManager.Task::(prepare) Task=`aec435e2-670f-4480-8640-6e6dbed75dd9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000822712', 'lastCheck': '5.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:22,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`aec435e2-670f-4480-8640-6e6dbed75dd9`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:22,892::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:22,892::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:22,892::task::993::Storage.TaskManager.Task::(_decref) Task=`aec435e2-670f-4480-8640-6e6dbed75dd9`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:27:22,912::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:22,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0184224 s, 55.6 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:27:23,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`f528bd91-1474-45eb-b18e-077f6143348b`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:27:23,298::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:27:23,299::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:23,300::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:23,300::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:23,300::__init__::232::IOProcess::(_processLogs) (7056) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:27:23,301::__init__::232::IOProcess::(_processLogs) DEBUG|(7056) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:23,302::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:23,302::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:23,302::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:23,303::__init__::232::IOProcess::(_processLogs) (7057) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:27:23,303::__init__::232::IOProcess::(_processLogs) (7057) Queuing response >Thread-122::INFO::2014-08-11 15:27:23,304::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:27:23,304::task::1191::Storage.TaskManager.Task::(prepare) Task=`f528bd91-1474-45eb-b18e-077f6143348b`::finished: {'truesize': '1169743872', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:27:23,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`f528bd91-1474-45eb-b18e-077f6143348b`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:27:23,304::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:27:23,305::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:27:23,305::task::993::Storage.TaskManager.Task::(_decref) Task=`f528bd91-1474-45eb-b18e-077f6143348b`::ref 0 aborting False >VM Channels Listener::DEBUG::2014-08-11 15:27:23,528::vmchannels::96::vds::(_handle_timeouts) Timeout on fileno 14. >Dummy-33::DEBUG::2014-08-11 15:27:24,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:24,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135645 s, 75.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:25,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:25,686::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:25,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:27,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:27,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013186 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:27,674::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:27,675::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:27:27,676::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:27:27,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:27,696::__init__::232::IOProcess::(_processLogs) (7058) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:27:27,696::__init__::232::IOProcess::(_processLogs) (7058) Queuing response >Thread-16::DEBUG::2014-08-11 15:27:27,698::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00046259 s, 984 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:27,699::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:27,699::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:27,700::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:27,700::__init__::232::IOProcess::(_processLogs) (7059) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:27:27,701::__init__::232::IOProcess::(_processLogs) DEBUG|(7059) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:27,702::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:27,702::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:27,703::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:27,703::__init__::232::IOProcess::(_processLogs) (7060) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:27,704::__init__::232::IOProcess::(_processLogs) DEBUG|(7060) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:27,704::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:27,705::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:27,705::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:27,706::__init__::232::IOProcess::(_processLogs) (7061) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:27,706::__init__::232::IOProcess::(_processLogs) (7061) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:27,706::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:27,707::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:27,707::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:27,707::__init__::232::IOProcess::(_processLogs) (7062) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:27,707::__init__::232::IOProcess::(_processLogs) DEBUG|(7062) Queuing response >Dummy-33::DEBUG::2014-08-11 15:27:29,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:29,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132743 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:30,381::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:27:30,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`53d8a449-9d25-433f-a0e4-1c7eac18f68c`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:30,382::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:27:30,405::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:30,405::task::1191::Storage.TaskManager.Task::(prepare) Task=`53d8a449-9d25-433f-a0e4-1c7eac18f68c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:30,405::task::595::Storage.TaskManager.Task::(_updateState) Task=`53d8a449-9d25-433f-a0e4-1c7eac18f68c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:30,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:30,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:30,406::task::993::Storage.TaskManager.Task::(_decref) Task=`53d8a449-9d25-433f-a0e4-1c7eac18f68c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:27:30,418::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:27:30,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`577524c4-b42c-404d-8eae-14501ae13124`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:27:30,419::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:27:30,420::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d7a7de8e-cd8d-4a0c-8486-4753fbf3e815`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:27:30,420::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:27:30,420::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:27:30,421::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d7a7de8e-cd8d-4a0c-8486-4753fbf3e815`::Granted request >Thread-34::DEBUG::2014-08-11 15:27:30,421::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`577524c4-b42c-404d-8eae-14501ae13124`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:27:30,421::task::993::Storage.TaskManager.Task::(_decref) Task=`577524c4-b42c-404d-8eae-14501ae13124`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:27:30,443::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:30,443::task::1191::Storage.TaskManager.Task::(prepare) Task=`577524c4-b42c-404d-8eae-14501ae13124`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:30,443::task::595::Storage.TaskManager.Task::(_updateState) Task=`577524c4-b42c-404d-8eae-14501ae13124`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:27:30,443::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:27:30,444::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:27:30,444::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:27:30,444::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:27:30,445::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:27:30,445::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:27:30,445::task::993::Storage.TaskManager.Task::(_decref) Task=`577524c4-b42c-404d-8eae-14501ae13124`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:30,691::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:30,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:30,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:31,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:31,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133145 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:27:33,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:33,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.014728 s, 69.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:27:35,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:35,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133653 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:35,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:35,701::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:35,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:37,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:37,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136246 s, 75.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:37,710::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:27:37,710::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:37,710::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:37,711::__init__::232::IOProcess::(_processLogs) (7063) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:27:37,711::__init__::232::IOProcess::(_processLogs) (7063) Queuing response >Thread-16::DEBUG::2014-08-11 15:27:37,712::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:27:37,732::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000518614 s, 877 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:37,733::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:37,734::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:27:37,734::__init__::232::IOProcess::(_processLogs) (7064) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:37,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:37,736::__init__::232::IOProcess::(_processLogs) (7065) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:37,736::__init__::232::IOProcess::(_processLogs) (7065) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:37,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:37,738::__init__::232::IOProcess::(_processLogs) (7066) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:37,738::__init__::232::IOProcess::(_processLogs) (7066) Queuing response >Thread-14::DEBUG::2014-08-11 15:27:37,739::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:27:37,739::__init__::232::IOProcess::(_processLogs) (7067) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:37,739::__init__::232::IOProcess::(_processLogs) (7067) Queuing response >Thread-115::DEBUG::2014-08-11 15:27:38,021::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7ec621e-7203-4263-a873-36a768cc1388`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:38,021::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:27:38,022::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000518614', 'lastCheck': '0.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:38,022::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7ec621e-7203-4263-a873-36a768cc1388`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000518614', 'lastCheck': '0.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:38,022::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7ec621e-7203-4263-a873-36a768cc1388`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:38,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:38,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:38,023::task::993::Storage.TaskManager.Task::(_decref) Task=`d7ec621e-7203-4263-a873-36a768cc1388`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:27:38,043::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:39,341::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:39,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132871 s, 77.1 MB/s\n'; <rc> = 0 >VM Channels Listener::DEBUG::2014-08-11 15:27:40,169::guestagent::279::vm.Vm::(_handleMessage) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::username: u'None' >Thread-115::DEBUG::2014-08-11 15:27:40,459::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:27:40,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`68477ac1-6b33-43c6-bf0d-9d97bc6b1c23`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:40,460::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:27:40,474::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:40,541::task::1191::Storage.TaskManager.Task::(prepare) Task=`68477ac1-6b33-43c6-bf0d-9d97bc6b1c23`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:40,541::task::595::Storage.TaskManager.Task::(_updateState) Task=`68477ac1-6b33-43c6-bf0d-9d97bc6b1c23`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:40,541::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:40,541::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:40,542::task::993::Storage.TaskManager.Task::(_decref) Task=`68477ac1-6b33-43c6-bf0d-9d97bc6b1c23`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:27:40,555::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:27:40,555::task::595::Storage.TaskManager.Task::(_updateState) Task=`20294c0a-bce6-4028-9af9-c6a5b11d53e7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:27:40,556::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:27:40,557::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`790ddd0d-f82c-4bb8-aadc-a598f23b1b87`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:27:40,557::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:27:40,557::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:27:40,558::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`790ddd0d-f82c-4bb8-aadc-a598f23b1b87`::Granted request >Thread-34::DEBUG::2014-08-11 15:27:40,558::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`20294c0a-bce6-4028-9af9-c6a5b11d53e7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:27:40,559::task::993::Storage.TaskManager.Task::(_decref) Task=`20294c0a-bce6-4028-9af9-c6a5b11d53e7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:27:40,579::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:40,579::task::1191::Storage.TaskManager.Task::(prepare) Task=`20294c0a-bce6-4028-9af9-c6a5b11d53e7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:40,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`20294c0a-bce6-4028-9af9-c6a5b11d53e7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:27:40,580::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:27:40,580::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:27:40,580::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:27:40,581::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:27:40,581::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:27:40,581::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:27:40,582::task::993::Storage.TaskManager.Task::(_decref) Task=`20294c0a-bce6-4028-9af9-c6a5b11d53e7`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:40,704::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:40,705::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:40,706::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:41,399::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:41,433::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137742 s, 74.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:27:43,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:43,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131547 s, 77.8 MB/s\n'; <rc> = 0 >VM Channels Listener::INFO::2014-08-11 15:27:44,767::guestagent::178::vm.Vm::(_handleAPIVersion) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Guest API version changed from 0 to 1 >Dummy-33::DEBUG::2014-08-11 15:27:45,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:45,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132164 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:45,712::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:45,714::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:45,715::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:47,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:47,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013357 s, 76.7 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:27:47,741::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:27:47,743::__init__::232::IOProcess::(_processLogs) (7068) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:27:47,751::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:27:47,752::__init__::232::IOProcess::(_processLogs) (7069) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 15:27:47,753::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:27:47,754::__init__::232::IOProcess::(_processLogs) (7070) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 15:27:47,755::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:27:47,756::__init__::232::IOProcess::(_processLogs) (7071) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:47,757::__init__::232::IOProcess::(_processLogs) (7072) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:27:47,758::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:27:47,758::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:27:47,758::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:27:47,759::__init__::232::IOProcess::(_processLogs) (7073) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:27:47,760::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:27:47,779::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428754 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:47,781::__init__::232::IOProcess::(_processLogs) (7074) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:27:47,782::__init__::232::IOProcess::(_processLogs) (7075) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:47,783::__init__::232::IOProcess::(_processLogs) (7076) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:47,784::__init__::232::IOProcess::(_processLogs) (7077) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:27:49,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:49,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133584 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:50,594::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:27:50,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5ff2a79-3fb8-4481-a6bd-ae76ccba0a54`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:50,640::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:27:50,653::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:50,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5ff2a79-3fb8-4481-a6bd-ae76ccba0a54`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:27:50,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5ff2a79-3fb8-4481-a6bd-ae76ccba0a54`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:50,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:50,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:50,654::task::993::Storage.TaskManager.Task::(_decref) Task=`b5ff2a79-3fb8-4481-a6bd-ae76ccba0a54`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:27:50,665::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:27:50,666::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f47ffff-0c9d-42ad-8d7e-d7034bfd03cf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:27:50,667::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:27:50,667::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f994ec91-5863-4b5f-8264-fbeae05dfea8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:27:50,668::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:27:50,668::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:27:50,668::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f994ec91-5863-4b5f-8264-fbeae05dfea8`::Granted request >Thread-34::DEBUG::2014-08-11 15:27:50,669::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9f47ffff-0c9d-42ad-8d7e-d7034bfd03cf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:27:50,669::task::993::Storage.TaskManager.Task::(_decref) Task=`9f47ffff-0c9d-42ad-8d7e-d7034bfd03cf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:27:50,689::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:50,689::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f47ffff-0c9d-42ad-8d7e-d7034bfd03cf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:27:50,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f47ffff-0c9d-42ad-8d7e-d7034bfd03cf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:27:50,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:27:50,690::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:27:50,690::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:27:50,691::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:27:50,691::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:27:50,691::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:27:50,691::task::993::Storage.TaskManager.Task::(_decref) Task=`9f47ffff-0c9d-42ad-8d7e-d7034bfd03cf`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:50,722::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:50,723::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:50,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:51,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:51,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013346 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:27:53,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`e94e22c6-fde1-459b-80ab-ef340820b64c`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:27:53,137::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:27:53,137::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428754', 'lastCheck': '5.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:53,137::task::1191::Storage.TaskManager.Task::(prepare) Task=`e94e22c6-fde1-459b-80ab-ef340820b64c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428754', 'lastCheck': '5.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:27:53,138::task::595::Storage.TaskManager.Task::(_updateState) Task=`e94e22c6-fde1-459b-80ab-ef340820b64c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:27:53,138::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:27:53,138::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:27:53,138::task::993::Storage.TaskManager.Task::(_decref) Task=`e94e22c6-fde1-459b-80ab-ef340820b64c`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:27:53,157::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:53,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:53,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132073 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:55,730::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:55,732::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:27:55,733::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:27:55,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:55,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132791 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:57,786::__init__::232::IOProcess::(_processLogs) (7078) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:27:57,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:27:57,806::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423354 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:27:57,808::__init__::232::IOProcess::(_processLogs) (7079) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:27:57,809::__init__::232::IOProcess::(_processLogs) (7080) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:57,810::__init__::232::IOProcess::(_processLogs) (7081) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:27:57,811::__init__::232::IOProcess::(_processLogs) (7082) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:27:57,848::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:57,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0167547 s, 61.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:27:59,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:27:59,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133748 s, 76.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:00,703::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-115::DEBUG::2014-08-11 15:28:00,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`706b14a7-3b33-4a44-bd16-3fc2acf3089c`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:00,705::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:28:00,717::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:00,718::task::1191::Storage.TaskManager.Task::(prepare) Task=`706b14a7-3b33-4a44-bd16-3fc2acf3089c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:00,718::task::595::Storage.TaskManager.Task::(_updateState) Task=`706b14a7-3b33-4a44-bd16-3fc2acf3089c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:00,718::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:00,718::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:00,719::task::993::Storage.TaskManager.Task::(_decref) Task=`706b14a7-3b33-4a44-bd16-3fc2acf3089c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:28:00,730::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-34::DEBUG::2014-08-11 15:28:00,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`4af20817-1a77-45ce-baa6-e84551473040`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:28:00,730::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:00,740::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:28:00,743::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1fef176d-69a6-416c-83f9-a5c91b569315`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:28:00,745::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:00,746::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:28:00,747::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:28:00,748::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1fef176d-69a6-416c-83f9-a5c91b569315`::Granted request >Thread-34::DEBUG::2014-08-11 15:28:00,750::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4af20817-1a77-45ce-baa6-e84551473040`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:28:00,752::task::993::Storage.TaskManager.Task::(_decref) Task=`4af20817-1a77-45ce-baa6-e84551473040`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:00,752::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:28:00,775::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:00,776::task::1191::Storage.TaskManager.Task::(prepare) Task=`4af20817-1a77-45ce-baa6-e84551473040`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959358976', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:00,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`4af20817-1a77-45ce-baa6-e84551473040`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:28:00,776::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:28:00,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:28:00,777::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:28:00,777::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:28:00,777::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:28:00,778::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:28:00,778::task::993::Storage.TaskManager.Task::(_decref) Task=`4af20817-1a77-45ce-baa6-e84551473040`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:28:01,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:02,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0347017 s, 29.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:04,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:04,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132362 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:05,763::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:05,764::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:05,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:06,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:06,299::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132761 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:07,813::__init__::232::IOProcess::(_processLogs) (7083) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:28:07,814::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:28:07,834::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000751883 s, 605 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:07,835::__init__::232::IOProcess::(_processLogs) (7084) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:07,837::__init__::232::IOProcess::(_processLogs) (7085) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:07,838::__init__::232::IOProcess::(_processLogs) (7086) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:07,839::__init__::232::IOProcess::(_processLogs) (7087) Got request for method 'access' >Thread-115::DEBUG::2014-08-11 15:28:08,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ce9e3d4-e97a-4ca4-aced-da2c4b17ca71`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:08,254::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:28:08,254::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000751883', 'lastCheck': '0.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:08,254::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ce9e3d4-e97a-4ca4-aced-da2c4b17ca71`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000751883', 'lastCheck': '0.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:08,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ce9e3d4-e97a-4ca4-aced-da2c4b17ca71`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:08,255::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:08,255::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:08,255::task::993::Storage.TaskManager.Task::(_decref) Task=`1ce9e3d4-e97a-4ca4-aced-da2c4b17ca71`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:28:08,278::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:08,321::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:08,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132231 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:10,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:10,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132676 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:10,790::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:28:10,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`af10ea7d-01d1-479f-8dd9-15d94075d97f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:10,841::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:10,848::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:10,850::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:10,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:28:10,854::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:10,854::task::1191::Storage.TaskManager.Task::(prepare) Task=`af10ea7d-01d1-479f-8dd9-15d94075d97f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:10,855::task::595::Storage.TaskManager.Task::(_updateState) Task=`af10ea7d-01d1-479f-8dd9-15d94075d97f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:10,855::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:10,855::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:10,855::task::993::Storage.TaskManager.Task::(_decref) Task=`af10ea7d-01d1-479f-8dd9-15d94075d97f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:28:10,866::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:28:10,866::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a5d38d9-e123-45e7-b892-826f4317330d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:28:10,867::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:28:10,867::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0c4a853e-f69a-4cf1-a69a-0786266eba1c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:28:10,868::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:28:10,868::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:28:10,868::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0c4a853e-f69a-4cf1-a69a-0786266eba1c`::Granted request >Thread-34::DEBUG::2014-08-11 15:28:10,869::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6a5d38d9-e123-45e7-b892-826f4317330d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:28:10,869::task::993::Storage.TaskManager.Task::(_decref) Task=`6a5d38d9-e123-45e7-b892-826f4317330d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:28:10,889::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:10,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`6a5d38d9-e123-45e7-b892-826f4317330d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:10,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`6a5d38d9-e123-45e7-b892-826f4317330d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:28:10,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:28:10,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:28:10,890::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:28:10,891::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:28:10,891::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:28:10,891::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:28:10,891::task::993::Storage.TaskManager.Task::(_decref) Task=`6a5d38d9-e123-45e7-b892-826f4317330d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:28:12,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:12,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013474 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:14,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:14,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131964 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:15,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:15,856::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:15,941::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:16,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:16,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132224 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:17,841::__init__::232::IOProcess::(_processLogs) (7088) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:28:17,842::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:28:17,860::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042386 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:17,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:17,862::__init__::232::IOProcess::(_processLogs) (7089) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:17,863::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:17,863::__init__::232::IOProcess::(_processLogs) (7090) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:17,863::__init__::232::IOProcess::(_processLogs) DEBUG|(7090) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:17,864::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:17,864::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:17,865::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:17,865::__init__::232::IOProcess::(_processLogs) DEBUG|(7091) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:17,865::__init__::232::IOProcess::(_processLogs) (7091) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:17,866::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:17,866::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:17,867::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:17,867::__init__::232::IOProcess::(_processLogs) DEBUG|(7092) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:17,867::__init__::232::IOProcess::(_processLogs) (7092) Queuing response >Dummy-33::DEBUG::2014-08-11 15:28:18,597::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:18,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132031 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:20,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:20,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133589 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:20,903::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:28:20,904::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f6dce45-19fc-4b18-a4f0-1619cd36b36a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:20,941::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:20,949::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:20,950::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:20,951::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:28:20,954::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:20,954::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f6dce45-19fc-4b18-a4f0-1619cd36b36a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:20,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f6dce45-19fc-4b18-a4f0-1619cd36b36a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:20,954::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:20,955::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:20,955::task::993::Storage.TaskManager.Task::(_decref) Task=`2f6dce45-19fc-4b18-a4f0-1619cd36b36a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:28:20,967::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:28:20,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4991ef1-71a7-4cf0-8f69-31efd7f3cd53`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:28:20,970::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:28:20,972::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6da8debf-f601-4ec3-b628-92fa141a9011`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:28:20,973::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:28:20,974::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:28:20,975::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6da8debf-f601-4ec3-b628-92fa141a9011`::Granted request >Thread-34::DEBUG::2014-08-11 15:28:20,979::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f4991ef1-71a7-4cf0-8f69-31efd7f3cd53`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:28:20,979::task::993::Storage.TaskManager.Task::(_decref) Task=`f4991ef1-71a7-4cf0-8f69-31efd7f3cd53`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:28:20,999::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:21,000::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4991ef1-71a7-4cf0-8f69-31efd7f3cd53`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:21,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4991ef1-71a7-4cf0-8f69-31efd7f3cd53`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:28:21,000::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:28:21,000::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:28:21,001::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:28:21,001::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:28:21,001::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:28:21,001::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:28:21,002::task::993::Storage.TaskManager.Task::(_decref) Task=`f4991ef1-71a7-4cf0-8f69-31efd7f3cd53`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:28:22,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:22,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132782 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:23,368::task::595::Storage.TaskManager.Task::(_updateState) Task=`75eab98c-3f2a-4b2b-ba7e-c96a7262b893`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:23,368::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:28:23,369::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042386', 'lastCheck': '5.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:23,369::task::1191::Storage.TaskManager.Task::(prepare) Task=`75eab98c-3f2a-4b2b-ba7e-c96a7262b893`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00042386', 'lastCheck': '5.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:23,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`75eab98c-3f2a-4b2b-ba7e-c96a7262b893`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:23,369::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:23,370::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:23,370::task::993::Storage.TaskManager.Task::(_decref) Task=`75eab98c-3f2a-4b2b-ba7e-c96a7262b893`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:28:23,389::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-122::DEBUG::2014-08-11 15:28:23,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b0bf71d-46f6-424c-8b6c-e3b500fab098`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:28:23,446::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:28:23,446::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:23,447::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:23,447::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:23,447::__init__::232::IOProcess::(_processLogs) (7093) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:28:23,448::__init__::232::IOProcess::(_processLogs) DEBUG|(7093) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:23,449::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:23,449::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:23,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:23,449::__init__::232::IOProcess::(_processLogs) DEBUG|(7094) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:28:23,450::__init__::232::IOProcess::(_processLogs) (7094) Queuing response >Thread-122::INFO::2014-08-11 15:28:23,450::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:28:23,450::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b0bf71d-46f6-424c-8b6c-e3b500fab098`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:28:23,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b0bf71d-46f6-424c-8b6c-e3b500fab098`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:28:23,451::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:28:23,451::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:28:23,451::task::993::Storage.TaskManager.Task::(_decref) Task=`1b0bf71d-46f6-424c-8b6c-e3b500fab098`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:28:24,764::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:24,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0192822 s, 53.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:25,958::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:25,959::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:26,041::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:26,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:26,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132979 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:27,869::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:27,869::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:27,870::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:27,870::__init__::232::IOProcess::(_processLogs) (7095) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:27,871::__init__::232::IOProcess::(_processLogs) DEBUG|(7095) Queuing response >Thread-16::DEBUG::2014-08-11 15:28:27,872::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:28:27,890::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418746 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:27,891::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:27,891::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:27,892::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:27,892::__init__::232::IOProcess::(_processLogs) (7096) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:27,892::__init__::232::IOProcess::(_processLogs) DEBUG|(7096) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:27,893::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:27,894::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:27,894::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:27,894::__init__::232::IOProcess::(_processLogs) (7097) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:27,895::__init__::232::IOProcess::(_processLogs) DEBUG|(7097) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:27,896::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:27,896::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:27,896::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:27,897::__init__::232::IOProcess::(_processLogs) (7098) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:27,897::__init__::232::IOProcess::(_processLogs) (7098) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:27,898::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:27,898::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:27,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:27,899::__init__::232::IOProcess::(_processLogs) (7099) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:27,899::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7099) Queuing response >Dummy-33::DEBUG::2014-08-11 15:28:28,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:28,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133003 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:30,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:30,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133922 s, 76.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:31,014::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:28:31,041::task::595::Storage.TaskManager.Task::(_updateState) Task=`16d0aa36-9be6-4046-968f-189e8ad56607`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:31,041::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:31,049::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:31,052::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:31,053::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:28:31,055::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:31,055::task::1191::Storage.TaskManager.Task::(prepare) Task=`16d0aa36-9be6-4046-968f-189e8ad56607`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:31,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`16d0aa36-9be6-4046-968f-189e8ad56607`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:31,056::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:31,056::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:31,056::task::993::Storage.TaskManager.Task::(_decref) Task=`16d0aa36-9be6-4046-968f-189e8ad56607`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:28:31,066::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:28:31,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`f56ddd5a-0121-4456-90e3-aa2f77e3d4fa`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:28:31,067::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:28:31,068::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ca0e5cec-2f6a-4ee1-a3f4-3824d37d7377`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:28:31,068::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:28:31,069::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:28:31,069::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ca0e5cec-2f6a-4ee1-a3f4-3824d37d7377`::Granted request >Thread-34::DEBUG::2014-08-11 15:28:31,071::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f56ddd5a-0121-4456-90e3-aa2f77e3d4fa`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:28:31,072::task::993::Storage.TaskManager.Task::(_decref) Task=`f56ddd5a-0121-4456-90e3-aa2f77e3d4fa`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:28:31,092::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:31,092::task::1191::Storage.TaskManager.Task::(prepare) Task=`f56ddd5a-0121-4456-90e3-aa2f77e3d4fa`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:31,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`f56ddd5a-0121-4456-90e3-aa2f77e3d4fa`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:28:31,093::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:28:31,093::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:28:31,093::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:28:31,094::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:28:31,094::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:28:31,094::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:28:31,094::task::993::Storage.TaskManager.Task::(_decref) Task=`f56ddd5a-0121-4456-90e3-aa2f77e3d4fa`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:28:32,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:33,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.023762 s, 43.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:35,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:35,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132335 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:36,060::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:36,061::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:36,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:37,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:37,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132437 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:37,901::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:37,901::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:37,902::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:37,902::__init__::232::IOProcess::(_processLogs) (7100) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:37,903::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7100) Queuing response >Thread-16::DEBUG::2014-08-11 15:28:37,904::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:28:37,922::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418697 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:37,924::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:37,924::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:37,924::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:37,924::__init__::232::IOProcess::(_processLogs) (7101) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:37,925::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7101) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:37,927::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:37,927::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:37,928::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:37,928::__init__::232::IOProcess::(_processLogs) (7102) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:37,929::__init__::232::IOProcess::(_processLogs) (7102) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:37,929::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:37,929::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:37,930::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:37,930::__init__::232::IOProcess::(_processLogs) (7103) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:37,931::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7103) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:37,931::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:37,932::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:37,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:37,933::__init__::232::IOProcess::(_processLogs) (7104) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:37,933::__init__::232::IOProcess::(_processLogs) (7104) Queuing response >Thread-115::DEBUG::2014-08-11 15:28:38,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba057a73-6244-4150-aac8-4a1123f31114`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:38,516::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:28:38,516::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418697', 'lastCheck': '0.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:38,516::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba057a73-6244-4150-aac8-4a1123f31114`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418697', 'lastCheck': '0.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:38,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba057a73-6244-4150-aac8-4a1123f31114`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:38,517::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:38,517::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:38,517::task::993::Storage.TaskManager.Task::(_decref) Task=`ba057a73-6244-4150-aac8-4a1123f31114`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:28:38,537::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:39,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:39,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131942 s, 77.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:41,107::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-115::DEBUG::2014-08-11 15:28:41,108::task::595::Storage.TaskManager.Task::(_updateState) Task=`8656e378-a907-44ae-a34d-f0e98a0f35fd`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:41,140::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:41,149::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:41,150::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:41,151::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:28:41,153::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:41,153::task::1191::Storage.TaskManager.Task::(prepare) Task=`8656e378-a907-44ae-a34d-f0e98a0f35fd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:41,154::task::595::Storage.TaskManager.Task::(_updateState) Task=`8656e378-a907-44ae-a34d-f0e98a0f35fd`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:41,154::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:41,154::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:41,154::task::993::Storage.TaskManager.Task::(_decref) Task=`8656e378-a907-44ae-a34d-f0e98a0f35fd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:28:41,165::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 15:28:41,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca41ae7d-5ce0-4109-8a67-081fca0ca39b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:28:41,166::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:28:41,167::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6b3ee811-b23d-4ee5-ad13-e9c8d5737e71`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:28:41,167::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:28:41,168::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:28:41,168::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6b3ee811-b23d-4ee5-ad13-e9c8d5737e71`::Granted request >Thread-34::DEBUG::2014-08-11 15:28:41,168::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ca41ae7d-5ce0-4109-8a67-081fca0ca39b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:28:41,169::task::993::Storage.TaskManager.Task::(_decref) Task=`ca41ae7d-5ce0-4109-8a67-081fca0ca39b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:28:41,188::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:41,189::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca41ae7d-5ce0-4109-8a67-081fca0ca39b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:41,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca41ae7d-5ce0-4109-8a67-081fca0ca39b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:28:41,189::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:28:41,189::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:28:41,190::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:28:41,190::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:28:41,190::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:28:41,190::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:28:41,191::task::993::Storage.TaskManager.Task::(_decref) Task=`ca41ae7d-5ce0-4109-8a67-081fca0ca39b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:28:41,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:41,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131946 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:43,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:43,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132099 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:45,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:45,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134506 s, 76.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:46,158::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:46,159::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:46,241::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:47,395::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:47,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132262 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:47,934::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:47,934::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:47,935::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:47,935::__init__::232::IOProcess::(_processLogs) (7105) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:47,937::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7105) Queuing response >Thread-16::DEBUG::2014-08-11 15:28:47,937::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:28:47,956::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000506929 s, 898 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:47,958::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:47,958::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:47,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:47,959::__init__::232::IOProcess::(_processLogs) (7106) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:47,959::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7106) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:47,961::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:47,961::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:47,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:47,962::__init__::232::IOProcess::(_processLogs) (7107) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:47,962::__init__::232::IOProcess::(_processLogs) (7107) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:47,963::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:47,963::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:47,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:47,964::__init__::232::IOProcess::(_processLogs) (7108) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:47,965::__init__::232::IOProcess::(_processLogs) (7108) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:47,965::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:47,966::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:47,966::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:47,967::__init__::232::IOProcess::(_processLogs) (7109) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:47,967::__init__::232::IOProcess::(_processLogs) (7109) Queuing response >Dummy-33::DEBUG::2014-08-11 15:28:49,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:49,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133106 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:51,203::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:28:51,204::task::595::Storage.TaskManager.Task::(_updateState) Task=`e395a833-dfd8-4047-a524-1fdd4b09786a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:51,240::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:51,248::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:51,250::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:51,251::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:28:51,253::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:51,253::task::1191::Storage.TaskManager.Task::(prepare) Task=`e395a833-dfd8-4047-a524-1fdd4b09786a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:28:51,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`e395a833-dfd8-4047-a524-1fdd4b09786a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:51,254::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:51,254::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:51,254::task::993::Storage.TaskManager.Task::(_decref) Task=`e395a833-dfd8-4047-a524-1fdd4b09786a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:28:51,265::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:28:51,265::task::595::Storage.TaskManager.Task::(_updateState) Task=`353751c7-1aa2-47ec-94f3-67637650b3bf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:28:51,265::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:28:51,266::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`06a6751c-cd5c-4e4c-9a71-104d1dfc2d86`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:28:51,267::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:28:51,267::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:28:51,267::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`06a6751c-cd5c-4e4c-9a71-104d1dfc2d86`::Granted request >Thread-34::DEBUG::2014-08-11 15:28:51,268::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`353751c7-1aa2-47ec-94f3-67637650b3bf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:28:51,268::task::993::Storage.TaskManager.Task::(_decref) Task=`353751c7-1aa2-47ec-94f3-67637650b3bf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:28:51,288::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:51,288::task::1191::Storage.TaskManager.Task::(prepare) Task=`353751c7-1aa2-47ec-94f3-67637650b3bf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:28:51,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`353751c7-1aa2-47ec-94f3-67637650b3bf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:28:51,288::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:28:51,289::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:28:51,289::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:28:51,289::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:28:51,290::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:28:51,290::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:28:51,290::task::993::Storage.TaskManager.Task::(_decref) Task=`353751c7-1aa2-47ec-94f3-67637650b3bf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:28:51,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:51,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134068 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:28:53,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:53,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134869 s, 75.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:28:53,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bd3fcc5-dcbf-46a7-9c82-0352593edcfd`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:28:53,638::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:28:53,639::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000506929', 'lastCheck': '5.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:53,639::task::1191::Storage.TaskManager.Task::(prepare) Task=`9bd3fcc5-dcbf-46a7-9c82-0352593edcfd`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000506929', 'lastCheck': '5.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:28:53,639::task::595::Storage.TaskManager.Task::(_updateState) Task=`9bd3fcc5-dcbf-46a7-9c82-0352593edcfd`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:28:53,639::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:28:53,640::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:28:53,640::task::993::Storage.TaskManager.Task::(_decref) Task=`9bd3fcc5-dcbf-46a7-9c82-0352593edcfd`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:28:53,660::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:55,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:55,645::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133443 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:56,256::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:56,257::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:28:56,341::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:28:57,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:57,701::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131902 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:57,968::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:57,968::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:57,969::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:57,969::__init__::232::IOProcess::(_processLogs) (7110) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:57,969::__init__::232::IOProcess::(_processLogs) (7110) Queuing response >Thread-16::DEBUG::2014-08-11 15:28:57,970::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:28:57,989::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419751 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:28:57,990::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:57,991::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:57,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:57,991::__init__::232::IOProcess::(_processLogs) (7111) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:28:57,992::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7111) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:57,993::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:57,994::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:57,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:57,994::__init__::232::IOProcess::(_processLogs) (7112) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:57,995::__init__::232::IOProcess::(_processLogs) (7112) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:57,995::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:57,996::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:57,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:57,996::__init__::232::IOProcess::(_processLogs) (7113) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:57,997::__init__::232::IOProcess::(_processLogs) (7113) Queuing response >Thread-14::DEBUG::2014-08-11 15:28:57,997::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:28:57,998::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:28:57,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:28:57,999::__init__::232::IOProcess::(_processLogs) (7114) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:28:57,999::__init__::232::IOProcess::(_processLogs) (7114) Queuing response >Dummy-33::DEBUG::2014-08-11 15:28:59,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:28:59,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133216 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:01,305::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:29:01,306::task::595::Storage.TaskManager.Task::(_updateState) Task=`4628e410-d2d1-45fc-be65-0aa94169be32`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:01,341::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:01,348::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:01,350::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:01,352::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:29:01,354::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:01,355::task::1191::Storage.TaskManager.Task::(prepare) Task=`4628e410-d2d1-45fc-be65-0aa94169be32`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:01,355::task::595::Storage.TaskManager.Task::(_updateState) Task=`4628e410-d2d1-45fc-be65-0aa94169be32`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:01,355::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:01,355::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:01,355::task::993::Storage.TaskManager.Task::(_decref) Task=`4628e410-d2d1-45fc-be65-0aa94169be32`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:29:01,367::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:29:01,367::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad7d9db0-5cc5-4f37-9132-3a3fc9d4e4c0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:29:01,368::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:29:01,368::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8991015-d451-4403-b26e-9a0115c6bd99`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:29:01,369::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:29:01,369::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:29:01,369::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8991015-d451-4403-b26e-9a0115c6bd99`::Granted request >Thread-34::DEBUG::2014-08-11 15:29:01,370::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ad7d9db0-5cc5-4f37-9132-3a3fc9d4e4c0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:29:01,370::task::993::Storage.TaskManager.Task::(_decref) Task=`ad7d9db0-5cc5-4f37-9132-3a3fc9d4e4c0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:29:01,390::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:01,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad7d9db0-5cc5-4f37-9132-3a3fc9d4e4c0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:01,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad7d9db0-5cc5-4f37-9132-3a3fc9d4e4c0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:29:01,391::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:29:01,391::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:29:01,392::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:29:01,392::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:29:01,392::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:29:01,392::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:29:01,393::task::993::Storage.TaskManager.Task::(_decref) Task=`ad7d9db0-5cc5-4f37-9132-3a3fc9d4e4c0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:29:01,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:01,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133271 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:03,833::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:03,865::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132431 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:05,888::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:05,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132225 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:06,358::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:06,359::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:06,443::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:07,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:07,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132995 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:08,000::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:08,000::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:08,000::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:08,001::__init__::232::IOProcess::(_processLogs) (7115) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:08,001::__init__::232::IOProcess::(_processLogs) (7115) Queuing response >Thread-16::DEBUG::2014-08-11 15:29:08,002::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:29:08,020::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000435065 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:08,021::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:08,021::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:08,022::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:08,022::__init__::232::IOProcess::(_processLogs) (7116) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:08,022::__init__::232::IOProcess::(_processLogs) (7116) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:08,023::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:08,024::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:08,024::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:08,025::__init__::232::IOProcess::(_processLogs) (7117) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:08,025::__init__::232::IOProcess::(_processLogs) (7117) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:08,025::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:08,026::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:08,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:08,027::__init__::232::IOProcess::(_processLogs) (7118) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:08,027::__init__::232::IOProcess::(_processLogs) (7118) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:08,027::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:08,028::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:08,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:08,029::__init__::232::IOProcess::(_processLogs) (7119) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:08,030::__init__::232::IOProcess::(_processLogs) (7119) Queuing response >Thread-115::DEBUG::2014-08-11 15:29:08,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`81eece5f-1b5d-4c25-9a76-be4a755dcbe0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:08,751::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:29:08,751::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435065', 'lastCheck': '0.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:08,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`81eece5f-1b5d-4c25-9a76-be4a755dcbe0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000435065', 'lastCheck': '0.7', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:08,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`81eece5f-1b5d-4c25-9a76-be4a755dcbe0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:08,752::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:08,752::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:08,752::task::993::Storage.TaskManager.Task::(_decref) Task=`81eece5f-1b5d-4c25-9a76-be4a755dcbe0`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:29:08,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:09,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:10,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133045 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:11,405::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:29:11,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`b97a60bb-d517-48a6-967d-9cb1f509fad1`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:11,443::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:11,450::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:11,452::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:11,453::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:29:11,455::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:11,456::task::1191::Storage.TaskManager.Task::(prepare) Task=`b97a60bb-d517-48a6-967d-9cb1f509fad1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:11,456::task::595::Storage.TaskManager.Task::(_updateState) Task=`b97a60bb-d517-48a6-967d-9cb1f509fad1`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:11,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:11,456::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:11,456::task::993::Storage.TaskManager.Task::(_decref) Task=`b97a60bb-d517-48a6-967d-9cb1f509fad1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:29:11,466::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:29:11,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8d9cf26-4241-4179-bc40-ec1948f1fae7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:29:11,467::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:29:11,468::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3ec69dad-8e9d-4df9-87aa-ad9ff5389685`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:29:11,468::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:29:11,469::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:29:11,469::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3ec69dad-8e9d-4df9-87aa-ad9ff5389685`::Granted request >Thread-34::DEBUG::2014-08-11 15:29:11,469::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e8d9cf26-4241-4179-bc40-ec1948f1fae7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:29:11,470::task::993::Storage.TaskManager.Task::(_decref) Task=`e8d9cf26-4241-4179-bc40-ec1948f1fae7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:29:11,489::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:11,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8d9cf26-4241-4179-bc40-ec1948f1fae7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:11,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8d9cf26-4241-4179-bc40-ec1948f1fae7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:29:11,490::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:29:11,490::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:29:11,491::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:29:11,491::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:29:11,491::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:29:11,492::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:29:11,492::task::993::Storage.TaskManager.Task::(_decref) Task=`e8d9cf26-4241-4179-bc40-ec1948f1fae7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:29:12,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:12,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132155 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:14,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:14,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013449 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:16,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:16,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132148 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:16,460::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:16,461::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:16,544::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:29:18,030::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:18,031::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:18,031::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:18,031::__init__::232::IOProcess::(_processLogs) (7120) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:18,032::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7120) Queuing response >Thread-16::DEBUG::2014-08-11 15:29:18,033::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:29:18,052::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428364 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:18,053::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:18,053::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:18,054::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:18,054::__init__::232::IOProcess::(_processLogs) (7121) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:18,055::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7121) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:18,056::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:18,056::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:18,057::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:18,057::__init__::232::IOProcess::(_processLogs) (7122) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:18,058::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7122) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:18,058::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:18,059::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:18,059::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:18,059::__init__::232::IOProcess::(_processLogs) (7123) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:18,060::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7123) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:18,060::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:18,061::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:18,061::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:18,061::__init__::232::IOProcess::(_processLogs) (7124) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:18,062::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7124) Queuing response >Dummy-33::DEBUG::2014-08-11 15:29:18,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:18,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134763 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:20,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:20,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134092 s, 76.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:21,506::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:29:21,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc6cc573-83e2-4a7f-ae4d-d7c7a7c5def3`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:21,543::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:21,546::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:21,548::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:21,549::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:29:21,557::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:21,557::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc6cc573-83e2-4a7f-ae4d-d7c7a7c5def3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:21,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc6cc573-83e2-4a7f-ae4d-d7c7a7c5def3`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:21,557::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:21,558::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:21,558::task::993::Storage.TaskManager.Task::(_decref) Task=`cc6cc573-83e2-4a7f-ae4d-d7c7a7c5def3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:29:21,572::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:29:21,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2942233-f373-4b8a-8231-ce34859f2ff9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:29:21,573::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:29:21,574::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8dd9b1a1-e827-4c48-a468-3e1add620aff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:29:21,574::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:29:21,574::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:29:21,574::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8dd9b1a1-e827-4c48-a468-3e1add620aff`::Granted request >Thread-34::DEBUG::2014-08-11 15:29:21,575::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f2942233-f373-4b8a-8231-ce34859f2ff9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:29:21,575::task::993::Storage.TaskManager.Task::(_decref) Task=`f2942233-f373-4b8a-8231-ce34859f2ff9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:29:21,595::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:21,595::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2942233-f373-4b8a-8231-ce34859f2ff9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959293440', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:21,595::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2942233-f373-4b8a-8231-ce34859f2ff9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:29:21,596::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:29:21,596::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:29:21,596::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:29:21,597::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:29:21,597::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:29:21,597::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:29:21,597::task::993::Storage.TaskManager.Task::(_decref) Task=`f2942233-f373-4b8a-8231-ce34859f2ff9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:29:22,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:22,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131511 s, 77.9 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:29:23,560::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec691d25-1367-4873-874a-ea55d4f69d2b`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:29:23,560::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:29:23,561::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:23,562::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:23,562::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:23,562::__init__::232::IOProcess::(_processLogs) (7125) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:29:23,563::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7125) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:23,564::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:23,565::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:23,565::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-122::INFO::2014-08-11 15:29:23,565::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:29:23,567::__init__::232::IOProcess::(_processLogs) (7126) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:29:23,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec691d25-1367-4873-874a-ea55d4f69d2b`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:29:23,568::__init__::232::IOProcess::(_processLogs) (7126) Queuing response >Thread-122::DEBUG::2014-08-11 15:29:23,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec691d25-1367-4873-874a-ea55d4f69d2b`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:29:23,569::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:29:23,569::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:29:23,570::task::993::Storage.TaskManager.Task::(_decref) Task=`ec691d25-1367-4873-874a-ea55d4f69d2b`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:29:23,866::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f646528-3426-45cf-8271-5605bea8d00a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:23,867::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:29:23,867::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428364', 'lastCheck': '5.8', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:23,867::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f646528-3426-45cf-8271-5605bea8d00a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428364', 'lastCheck': '5.8', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:23,867::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f646528-3426-45cf-8271-5605bea8d00a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:23,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:23,868::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:23,868::task::993::Storage.TaskManager.Task::(_decref) Task=`2f646528-3426-45cf-8271-5605bea8d00a`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:29:23,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:24,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:25,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.48661 s, 689 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:26,556::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:26,557::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:26,644::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:27,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:27,950::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132977 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:28,064::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:28,064::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:28,064::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:28,065::__init__::232::IOProcess::(_processLogs) (7127) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:28,065::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7127) Queuing response >Thread-16::DEBUG::2014-08-11 15:29:28,066::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:29:28,084::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416503 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:28,085::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:28,085::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:28,086::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:28,086::__init__::232::IOProcess::(_processLogs) (7128) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:28,086::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7128) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:28,088::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:28,088::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:28,088::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:28,088::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7129) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:28,089::__init__::232::IOProcess::(_processLogs) (7129) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:28,089::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:28,090::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:28,090::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:28,090::__init__::232::IOProcess::(_processLogs) (7130) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:28,091::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7130) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:28,091::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:28,092::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:28,092::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:28,092::__init__::232::IOProcess::(_processLogs) (7131) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:28,093::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7131) Queuing response >Dummy-33::DEBUG::2014-08-11 15:29:29,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:30,006::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133868 s, 76.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:31,611::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:29:31,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`7519f86a-673b-4b4a-a646-7b1eaea8aa0f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:31,612::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:29:31,626::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:31,626::task::1191::Storage.TaskManager.Task::(prepare) Task=`7519f86a-673b-4b4a-a646-7b1eaea8aa0f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:31,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`7519f86a-673b-4b4a-a646-7b1eaea8aa0f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:31,627::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:31,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:31,627::task::993::Storage.TaskManager.Task::(_decref) Task=`7519f86a-673b-4b4a-a646-7b1eaea8aa0f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:29:31,638::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:29:31,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`83173921-4094-47a2-8423-ad72b642959c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:29:31,639::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:29:31,640::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2a5338a6-fcbc-43d9-b22c-c8ddefefe6f1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:29:31,640::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:29:31,640::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:29:31,640::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2a5338a6-fcbc-43d9-b22c-c8ddefefe6f1`::Granted request >Thread-34::DEBUG::2014-08-11 15:29:31,641::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`83173921-4094-47a2-8423-ad72b642959c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:29:31,641::task::993::Storage.TaskManager.Task::(_decref) Task=`83173921-4094-47a2-8423-ad72b642959c`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:31,650::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:31,652::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:31,654::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:29:31,662::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959227904', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:31,662::task::1191::Storage.TaskManager.Task::(prepare) Task=`83173921-4094-47a2-8423-ad72b642959c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959227904', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:31,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`83173921-4094-47a2-8423-ad72b642959c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:29:31,663::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:29:31,663::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:29:31,663::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:29:31,664::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:29:31,664::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:29:31,664::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:29:31,664::task::993::Storage.TaskManager.Task::(_decref) Task=`83173921-4094-47a2-8423-ad72b642959c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:29:32,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:32,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305033 s, 33.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:34,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:34,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139017 s, 73.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:36,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:36,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132225 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:36,657::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:36,658::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:36,744::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:29:38,094::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:38,095::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:38,095::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:38,095::__init__::232::IOProcess::(_processLogs) (7132) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:38,096::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7132) Queuing response >Thread-16::DEBUG::2014-08-11 15:29:38,097::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:29:38,117::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428949 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:38,118::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:38,118::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:38,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:38,119::__init__::232::IOProcess::(_processLogs) (7133) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:38,119::__init__::232::IOProcess::(_processLogs) (7133) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:38,121::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:38,121::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:38,122::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:38,122::__init__::232::IOProcess::(_processLogs) (7134) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:38,123::__init__::232::IOProcess::(_processLogs) (7134) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:38,123::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:38,123::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:38,123::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:38,124::__init__::232::IOProcess::(_processLogs) (7135) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:38,124::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|(7135) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:38,125::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:38,126::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:38,126::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:38,127::__init__::232::IOProcess::(_processLogs) (7136) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:38,127::__init__::232::IOProcess::(_processLogs) (7136) Queuing response >Dummy-33::DEBUG::2014-08-11 15:29:38,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:38,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132654 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:38,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`85eecb9c-bed9-472e-aa97-336072d83c65`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:38,984::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:29:38,984::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428949', 'lastCheck': '0.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:38,985::task::1191::Storage.TaskManager.Task::(prepare) Task=`85eecb9c-bed9-472e-aa97-336072d83c65`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428949', 'lastCheck': '0.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:38,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`85eecb9c-bed9-472e-aa97-336072d83c65`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:38,985::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:38,985::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:38,985::task::993::Storage.TaskManager.Task::(_decref) Task=`85eecb9c-bed9-472e-aa97-336072d83c65`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:29:39,009::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:40,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:40,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134469 s, 76.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:41,677::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:29:41,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`48faac8e-62df-4a8a-a4be-063a9f7246a5`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:41,679::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:29:41,692::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:41,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`48faac8e-62df-4a8a-a4be-063a9f7246a5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:41,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`48faac8e-62df-4a8a-a4be-063a9f7246a5`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:41,693::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:41,693::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:41,693::task::993::Storage.TaskManager.Task::(_decref) Task=`48faac8e-62df-4a8a-a4be-063a9f7246a5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:29:41,703::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:29:41,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b06aba2-5d51-4870-b4b0-60bff64a00dd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:29:41,704::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:29:41,705::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`848603c8-a739-4bc0-9583-2daa4a01c733`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:29:41,705::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:29:41,706::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:29:41,706::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`848603c8-a739-4bc0-9583-2daa4a01c733`::Granted request >Thread-34::DEBUG::2014-08-11 15:29:41,707::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4b06aba2-5d51-4870-b4b0-60bff64a00dd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:29:41,707::task::993::Storage.TaskManager.Task::(_decref) Task=`4b06aba2-5d51-4870-b4b0-60bff64a00dd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:29:41,728::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:41,728::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b06aba2-5d51-4870-b4b0-60bff64a00dd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:41,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b06aba2-5d51-4870-b4b0-60bff64a00dd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:29:41,729::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:29:41,729::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:29:41,729::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:29:41,729::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:29:41,730::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:29:41,730::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:29:41,730::task::993::Storage.TaskManager.Task::(_decref) Task=`4b06aba2-5d51-4870-b4b0-60bff64a00dd`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:41,751::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:41,752::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:41,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:42,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:42,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137896 s, 74.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:44,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:44,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131462 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:46,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:46,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132641 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:46,758::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:46,760::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:46,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:29:48,128::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:48,128::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:48,129::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:48,129::__init__::232::IOProcess::(_processLogs) (7137) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:48,129::__init__::232::IOProcess::(_processLogs) (7137) Queuing response >Thread-16::DEBUG::2014-08-11 15:29:48,130::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:29:48,150::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041231 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:48,152::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:48,152::__init__::232::IOProcess::(_processLogs) (7133) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:48,152::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:48,153::__init__::232::IOProcess::(_processLogs) (7138) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:48,153::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7138) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:48,155::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:48,155::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:48,156::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:48,156::__init__::232::IOProcess::(_processLogs) (7139) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:48,156::__init__::232::IOProcess::(_processLogs) (7139) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:48,157::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:48,157::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:48,158::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:48,158::__init__::232::IOProcess::(_processLogs) (7140) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:48,158::__init__::232::IOProcess::(_processLogs) (7140) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:48,159::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:48,159::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:48,160::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:48,161::__init__::232::IOProcess::(_processLogs) (7141) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:48,161::__init__::232::IOProcess::(_processLogs) (7141) Queuing response >Dummy-33::DEBUG::2014-08-11 15:29:48,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:48,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133432 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:50,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:50,750::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135604 s, 75.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:51,744::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:29:51,745::task::595::Storage.TaskManager.Task::(_updateState) Task=`feefa4ed-12b8-498a-ad21-8746ebcc4f0b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:51,745::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:29:51,758::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:51,758::task::1191::Storage.TaskManager.Task::(prepare) Task=`feefa4ed-12b8-498a-ad21-8746ebcc4f0b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:29:51,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`feefa4ed-12b8-498a-ad21-8746ebcc4f0b`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:51,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:29:51,845::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:51,945::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:51,947::task::993::Storage.TaskManager.Task::(_decref) Task=`feefa4ed-12b8-498a-ad21-8746ebcc4f0b`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:51,950::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:51,951::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:29:51,960::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:29:51,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`31442e21-f556-44f9-972b-1164ae201aec`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:29:51,961::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:29:51,962::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`28255d4d-b436-4b70-946a-61730e92a56e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:29:51,962::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:29:51,962::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:29:51,963::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`28255d4d-b436-4b70-946a-61730e92a56e`::Granted request >Thread-34::DEBUG::2014-08-11 15:29:51,963::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`31442e21-f556-44f9-972b-1164ae201aec`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:29:51,963::task::993::Storage.TaskManager.Task::(_decref) Task=`31442e21-f556-44f9-972b-1164ae201aec`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:29:51,984::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:51,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`31442e21-f556-44f9-972b-1164ae201aec`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:29:51,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`31442e21-f556-44f9-972b-1164ae201aec`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:29:51,984::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:29:51,985::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:29:51,985::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:29:51,985::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:29:51,986::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:29:51,986::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:29:51,986::task::993::Storage.TaskManager.Task::(_decref) Task=`31442e21-f556-44f9-972b-1164ae201aec`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:29:52,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:52,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132359 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:29:54,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`60b1dbbf-b416-44f1-8223-617bc1738ba0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:29:54,104::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:29:54,104::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041231', 'lastCheck': '5.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:54,104::task::1191::Storage.TaskManager.Task::(prepare) Task=`60b1dbbf-b416-44f1-8223-617bc1738ba0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041231', 'lastCheck': '5.9', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:29:54,104::task::595::Storage.TaskManager.Task::(_updateState) Task=`60b1dbbf-b416-44f1-8223-617bc1738ba0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:29:54,105::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:29:54,105::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:29:54,105::task::993::Storage.TaskManager.Task::(_decref) Task=`60b1dbbf-b416-44f1-8223-617bc1738ba0`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:29:54,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:54,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:54,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132764 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:29:56,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:56,961::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:56,963::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:29:56,964::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:29:56,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133108 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:58,162::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:58,162::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:58,162::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:58,163::__init__::232::IOProcess::(_processLogs) (7142) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:58,163::__init__::232::IOProcess::(_processLogs) (7142) Queuing response >Thread-16::DEBUG::2014-08-11 15:29:58,164::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:29:58,182::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041753 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:29:58,184::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:58,184::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:58,184::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:58,185::__init__::232::IOProcess::(_processLogs) (7143) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:29:58,185::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7143) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:58,187::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:58,187::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:58,188::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:58,188::__init__::232::IOProcess::(_processLogs) (7144) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:58,188::__init__::232::IOProcess::(_processLogs) (7144) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:58,189::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:58,189::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:58,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:58,190::__init__::232::IOProcess::(_processLogs) (7145) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:58,191::__init__::232::IOProcess::(_processLogs) (7145) Queuing response >Thread-14::DEBUG::2014-08-11 15:29:58,191::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:29:58,192::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:29:58,192::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:29:58,193::__init__::232::IOProcess::(_processLogs) (7146) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:29:58,193::__init__::232::IOProcess::(_processLogs) (7146) Queuing response >Dummy-33::DEBUG::2014-08-11 15:29:59,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:29:59,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134189 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:01,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:01,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152366 s, 67.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:01,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:01,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:01,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:30:02,000::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:30:02,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`e12dd0e9-4fef-4324-b26e-e4dd0531e455`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:02,004::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:30:02,025::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:02,026::task::1191::Storage.TaskManager.Task::(prepare) Task=`e12dd0e9-4fef-4324-b26e-e4dd0531e455`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:02,027::task::595::Storage.TaskManager.Task::(_updateState) Task=`e12dd0e9-4fef-4324-b26e-e4dd0531e455`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:02,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:02,029::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:02,031::task::993::Storage.TaskManager.Task::(_decref) Task=`e12dd0e9-4fef-4324-b26e-e4dd0531e455`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:30:02,044::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:30:02,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`3dac2622-6f69-4495-b99b-a88f519e63f0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:30:02,048::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:30:02,052::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4145fd42-ba77-43e7-a83c-9ae52edd5383`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:30:02,053::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:30:02,055::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:30:02,057::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4145fd42-ba77-43e7-a83c-9ae52edd5383`::Granted request >Thread-34::DEBUG::2014-08-11 15:30:02,061::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3dac2622-6f69-4495-b99b-a88f519e63f0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:30:02,063::task::993::Storage.TaskManager.Task::(_decref) Task=`3dac2622-6f69-4495-b99b-a88f519e63f0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:30:02,125::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:02,127::task::1191::Storage.TaskManager.Task::(prepare) Task=`3dac2622-6f69-4495-b99b-a88f519e63f0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:02,128::task::595::Storage.TaskManager.Task::(_updateState) Task=`3dac2622-6f69-4495-b99b-a88f519e63f0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:30:02,129::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:30:02,130::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:30:02,132::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:30:02,134::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:30:02,135::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:30:02,136::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:30:02,137::task::993::Storage.TaskManager.Task::(_decref) Task=`3dac2622-6f69-4495-b99b-a88f519e63f0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:30:03,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:03,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013279 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:05,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:05,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132285 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:06,993::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:07,046::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:07,047::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:07,241::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:07,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131692 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:08,194::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:08,194::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:08,195::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:08,195::__init__::232::IOProcess::(_processLogs) (7147) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:08,195::__init__::232::IOProcess::(_processLogs) (7147) Queuing response >Thread-16::DEBUG::2014-08-11 15:30:08,196::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:30:08,214::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424652 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:08,216::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:08,216::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:08,216::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:08,217::__init__::232::IOProcess::(_processLogs) (7148) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:08,218::__init__::232::IOProcess::(_processLogs) (7148) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:08,219::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:08,219::__init__::232::IOProcess::(_processLogs) (7149) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:08,220::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:08,220::__init__::232::IOProcess::(_processLogs) (7150) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:08,221::__init__::232::IOProcess::(_processLogs) (7150) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:08,221::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:08,222::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:08,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:08,224::__init__::232::IOProcess::(_processLogs) (7151) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:08,224::__init__::232::IOProcess::(_processLogs) (7151) Queuing response >Thread-115::DEBUG::2014-08-11 15:30:09,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7ed161f-92b4-4898-abf0-3e8306ecbedf`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:09,221::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:30:09,222::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424652', 'lastCheck': '1.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:09,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7ed161f-92b4-4898-abf0-3e8306ecbedf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424652', 'lastCheck': '1.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:09,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7ed161f-92b4-4898-abf0-3e8306ecbedf`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:09,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:09,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:09,223::task::993::Storage.TaskManager.Task::(_decref) Task=`c7ed161f-92b4-4898-abf0-3e8306ecbedf`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:30:09,245::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:09,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:09,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133317 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:11,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:11,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131876 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:12,051::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:12,053::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:12,146::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:30:12,153::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:30:12,154::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce764896-db7d-4b1f-979f-42a256e80b4b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:12,154::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:30:12,167::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:12,167::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce764896-db7d-4b1f-979f-42a256e80b4b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:12,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce764896-db7d-4b1f-979f-42a256e80b4b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:12,168::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:12,168::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:12,168::task::993::Storage.TaskManager.Task::(_decref) Task=`ce764896-db7d-4b1f-979f-42a256e80b4b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:30:12,179::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:30:12,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`eeea4c59-9dac-4a1a-92ea-1ef2b66d98d2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:30:12,180::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:30:12,180::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`78ff2bd1-39bb-4260-9324-2a9df3539b04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:30:12,181::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:30:12,181::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:30:12,181::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`78ff2bd1-39bb-4260-9324-2a9df3539b04`::Granted request >Thread-34::DEBUG::2014-08-11 15:30:12,182::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`eeea4c59-9dac-4a1a-92ea-1ef2b66d98d2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:30:12,182::task::993::Storage.TaskManager.Task::(_decref) Task=`eeea4c59-9dac-4a1a-92ea-1ef2b66d98d2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:30:12,202::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:12,202::task::1191::Storage.TaskManager.Task::(prepare) Task=`eeea4c59-9dac-4a1a-92ea-1ef2b66d98d2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:12,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`eeea4c59-9dac-4a1a-92ea-1ef2b66d98d2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:30:12,202::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:30:12,203::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:30:12,203::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:30:12,203::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:30:12,204::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:30:12,204::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:30:12,204::task::993::Storage.TaskManager.Task::(_decref) Task=`eeea4c59-9dac-4a1a-92ea-1ef2b66d98d2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:30:13,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:13,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133275 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:15,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:15,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0221668 s, 46.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:17,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:17,155::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:17,156::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:17,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:17,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132411 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:18,224::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:18,225::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:18,225::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:18,225::__init__::232::IOProcess::(_processLogs) (7152) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:18,226::__init__::232::IOProcess::(_processLogs) (7152) Queuing response >Thread-16::DEBUG::2014-08-11 15:30:18,226::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:30:18,245::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417351 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:18,246::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:18,246::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:18,247::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:18,247::__init__::232::IOProcess::(_processLogs) (7153) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:18,247::__init__::232::IOProcess::(_processLogs) (7153) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:18,248::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:18,249::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:18,249::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:18,250::__init__::232::IOProcess::(_processLogs) (7154) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:18,250::__init__::232::IOProcess::(_processLogs) (7154) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:18,250::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:18,251::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:18,251::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:18,252::__init__::232::IOProcess::(_processLogs) (7155) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:18,252::__init__::232::IOProcess::(_processLogs) (7155) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:18,252::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:18,253::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:18,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:18,254::__init__::232::IOProcess::(_processLogs) (7156) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:18,254::__init__::232::IOProcess::(_processLogs) (7156) Queuing response >Dummy-33::DEBUG::2014-08-11 15:30:19,580::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:19,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135257 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:21,636::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:21,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134383 s, 76.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:22,163::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:22,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:30:22,216::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:30:22,249::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5dbce49-837c-4e54-82e2-8baaad551eab`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:22,251::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:22,251::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:30:22,265::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:22,266::task::1191::Storage.TaskManager.Task::(prepare) Task=`a5dbce49-837c-4e54-82e2-8baaad551eab`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:22,266::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5dbce49-837c-4e54-82e2-8baaad551eab`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:22,266::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:22,266::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:22,267::task::993::Storage.TaskManager.Task::(_decref) Task=`a5dbce49-837c-4e54-82e2-8baaad551eab`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:30:22,278::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:30:22,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`d203ca18-7ff2-4aea-b593-35acd50b7e31`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:30:22,279::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:30:22,279::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`628344d9-fe97-4e24-a2a3-a64315bd43cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:30:22,279::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:30:22,280::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:30:22,280::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`628344d9-fe97-4e24-a2a3-a64315bd43cf`::Granted request >Thread-34::DEBUG::2014-08-11 15:30:22,281::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d203ca18-7ff2-4aea-b593-35acd50b7e31`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:30:22,281::task::993::Storage.TaskManager.Task::(_decref) Task=`d203ca18-7ff2-4aea-b593-35acd50b7e31`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:30:22,301::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:22,301::task::1191::Storage.TaskManager.Task::(prepare) Task=`d203ca18-7ff2-4aea-b593-35acd50b7e31`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:22,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`d203ca18-7ff2-4aea-b593-35acd50b7e31`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:30:22,302::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:30:22,302::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:30:22,302::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:30:22,302::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:30:22,303::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:30:22,303::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:30:22,303::task::993::Storage.TaskManager.Task::(_decref) Task=`d203ca18-7ff2-4aea-b593-35acd50b7e31`::ref 0 aborting False >Thread-122::DEBUG::2014-08-11 15:30:23,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`a021cf97-0018-4b19-b49f-73226b527389`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:30:23,678::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:30:23,679::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:23,680::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:23,680::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:23,680::__init__::232::IOProcess::(_processLogs) (7157) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:30:23,681::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|(7157) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:23,682::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:23,683::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:23,684::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:23,684::__init__::232::IOProcess::(_processLogs) (7158) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:30:23,685::__init__::232::IOProcess::(_processLogs) (7158) Queuing response >Thread-122::INFO::2014-08-11 15:30:23,683::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:30:23,685::task::1191::Storage.TaskManager.Task::(prepare) Task=`a021cf97-0018-4b19-b49f-73226b527389`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:30:23,685::task::595::Storage.TaskManager.Task::(_updateState) Task=`a021cf97-0018-4b19-b49f-73226b527389`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:30:23,686::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:30:23,686::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:30:23,686::task::993::Storage.TaskManager.Task::(_decref) Task=`a021cf97-0018-4b19-b49f-73226b527389`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:30:23,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:23,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013676 s, 74.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:30:24,344::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5cd9e4c-788f-4e53-952e-620343949c08`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:24,345::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:30:24,345::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417351', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:24,346::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5cd9e4c-788f-4e53-952e-620343949c08`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417351', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:24,346::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5cd9e4c-788f-4e53-952e-620343949c08`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:24,346::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:24,346::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:24,346::task::993::Storage.TaskManager.Task::(_decref) Task=`b5cd9e4c-788f-4e53-952e-620343949c08`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:30:24,379::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:25,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:25,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134147 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:27,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:27,256::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:27,347::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:27,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:27,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131847 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:28,255::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:28,256::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:30:28,257::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:30:28,257::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:28,275::__init__::232::IOProcess::(_processLogs) (7159) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:28,276::__init__::232::IOProcess::(_processLogs) (7159) Queuing response >Thread-16::DEBUG::2014-08-11 15:30:28,277::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00059419 s, 766 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:28,278::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:28,278::__init__::232::IOProcess::(_processLogs) (7150) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:28,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:28,279::__init__::232::IOProcess::(_processLogs) (7160) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:28,280::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:28,280::__init__::232::IOProcess::(_processLogs) (7161) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:28,281::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:28,282::__init__::232::IOProcess::(_processLogs) (7162) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:28,282::__init__::232::IOProcess::(_processLogs) (7162) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:28,283::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:28,283::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:28,284::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:28,284::__init__::232::IOProcess::(_processLogs) (7163) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:28,284::__init__::232::IOProcess::(_processLogs) DEBUG|(7163) Queuing response >Dummy-33::DEBUG::2014-08-11 15:30:29,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:29,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133119 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:31,915::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:31,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134336 s, 76.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:30:32,317::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:30:32,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`93d597d8-e55a-47b2-99d1-41071b605912`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:32,347::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:32,359::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:30:32,365::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:32,366::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:30:32,367::task::1191::Storage.TaskManager.Task::(prepare) Task=`93d597d8-e55a-47b2-99d1-41071b605912`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:32,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`93d597d8-e55a-47b2-99d1-41071b605912`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:32,370::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:32,372::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:32,372::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:30:32,373::task::993::Storage.TaskManager.Task::(_decref) Task=`93d597d8-e55a-47b2-99d1-41071b605912`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:30:32,387::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:30:32,390::task::595::Storage.TaskManager.Task::(_updateState) Task=`31576fa4-ec69-4b7f-9e1d-03508f1e59d9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:30:32,391::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:30:32,396::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8671da3b-df1a-4ac2-8dbb-9d9c55e4c1d6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:30:32,397::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:30:32,400::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:30:32,401::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8671da3b-df1a-4ac2-8dbb-9d9c55e4c1d6`::Granted request >Thread-34::DEBUG::2014-08-11 15:30:32,405::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`31576fa4-ec69-4b7f-9e1d-03508f1e59d9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:30:32,406::task::993::Storage.TaskManager.Task::(_decref) Task=`31576fa4-ec69-4b7f-9e1d-03508f1e59d9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:30:32,467::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:32,468::task::1191::Storage.TaskManager.Task::(prepare) Task=`31576fa4-ec69-4b7f-9e1d-03508f1e59d9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:32,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`31576fa4-ec69-4b7f-9e1d-03508f1e59d9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:30:32,471::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:30:32,472::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:30:32,474::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:30:32,476::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:30:32,477::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:30:32,479::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:30:32,481::task::993::Storage.TaskManager.Task::(_decref) Task=`31576fa4-ec69-4b7f-9e1d-03508f1e59d9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:30:34,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:34,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132355 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:36,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:36,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132602 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:37,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:37,383::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:37,448::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:38,240::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:38,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132353 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:38,287::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:38,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:38,290::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:30:38,291::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:30:38,291::__init__::232::IOProcess::(_processLogs) (7164) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:38,310::__init__::232::IOProcess::(_processLogs) (7164) Queuing response >Thread-16::DEBUG::2014-08-11 15:30:38,319::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00670383 s, 67.9 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:38,320::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:38,321::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:38,321::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:38,321::__init__::232::IOProcess::(_processLogs) (7165) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:38,322::__init__::232::IOProcess::(_processLogs) DEBUG|(7165) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:38,323::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:38,324::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:38,324::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:38,325::__init__::232::IOProcess::(_processLogs) (7166) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:38,325::__init__::232::IOProcess::(_processLogs) (7166) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:38,325::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:38,326::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:38,326::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:38,326::__init__::232::IOProcess::(_processLogs) (7167) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:38,327::__init__::232::IOProcess::(_processLogs) DEBUG|(7167) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:38,327::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:38,328::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:38,328::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:38,329::__init__::232::IOProcess::(_processLogs) (7168) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:38,330::__init__::232::IOProcess::(_processLogs) (7168) Queuing response >Thread-115::DEBUG::2014-08-11 15:30:39,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`a82af821-fc62-4815-81e7-a03d17c9a0d0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:39,477::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:30:39,477::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00670383', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:39,477::task::1191::Storage.TaskManager.Task::(prepare) Task=`a82af821-fc62-4815-81e7-a03d17c9a0d0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00670383', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:39,478::task::595::Storage.TaskManager.Task::(_updateState) Task=`a82af821-fc62-4815-81e7-a03d17c9a0d0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:39,478::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:39,478::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:39,478::task::993::Storage.TaskManager.Task::(_decref) Task=`a82af821-fc62-4815-81e7-a03d17c9a0d0`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:30:39,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:40,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:40,352::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133847 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:42,375::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:42,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132479 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:42,454::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:42,455::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:42,456::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:30:42,498::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:30:42,499::task::595::Storage.TaskManager.Task::(_updateState) Task=`99d8f102-ebff-4a54-8498-a5c9cb5ad7f8`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:42,499::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:30:42,511::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:42,512::task::1191::Storage.TaskManager.Task::(prepare) Task=`99d8f102-ebff-4a54-8498-a5c9cb5ad7f8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:42,512::task::595::Storage.TaskManager.Task::(_updateState) Task=`99d8f102-ebff-4a54-8498-a5c9cb5ad7f8`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:42,512::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:42,512::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:42,513::task::993::Storage.TaskManager.Task::(_decref) Task=`99d8f102-ebff-4a54-8498-a5c9cb5ad7f8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:30:42,523::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:30:42,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`afc5a254-2328-426f-b5e9-b1424660d954`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:30:42,524::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:30:42,525::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ecc94b07-40c2-46cd-9717-e3d0974ed500`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:30:42,525::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:30:42,526::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:30:42,526::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ecc94b07-40c2-46cd-9717-e3d0974ed500`::Granted request >Thread-34::DEBUG::2014-08-11 15:30:42,526::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`afc5a254-2328-426f-b5e9-b1424660d954`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:30:42,527::task::993::Storage.TaskManager.Task::(_decref) Task=`afc5a254-2328-426f-b5e9-b1424660d954`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:30:42,546::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:42,547::task::1191::Storage.TaskManager.Task::(prepare) Task=`afc5a254-2328-426f-b5e9-b1424660d954`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:42,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`afc5a254-2328-426f-b5e9-b1424660d954`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:30:42,547::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:30:42,548::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:30:42,548::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:30:42,548::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:30:42,548::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:30:42,549::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:30:42,549::task::993::Storage.TaskManager.Task::(_decref) Task=`afc5a254-2328-426f-b5e9-b1424660d954`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:30:44,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:44,469::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0198903 s, 51.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:46,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:46,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013304 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:47,462::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:47,463::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:47,548::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:30:48,330::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:48,330::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:48,331::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:48,331::__init__::232::IOProcess::(_processLogs) (7169) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:48,331::__init__::232::IOProcess::(_processLogs) (7169) Queuing response >Thread-16::DEBUG::2014-08-11 15:30:48,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:30:48,351::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415614 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:48,352::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:48,352::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:48,352::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:48,353::__init__::232::IOProcess::(_processLogs) (7170) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:48,353::__init__::232::IOProcess::(_processLogs) DEBUG|(7170) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:48,355::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:48,355::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:48,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:48,356::__init__::232::IOProcess::(_processLogs) (7171) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:48,356::__init__::232::IOProcess::(_processLogs) (7171) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:48,357::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:48,357::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:48,358::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:48,358::__init__::232::IOProcess::(_processLogs) (7172) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:48,359::__init__::232::IOProcess::(_processLogs) (7172) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:48,359::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:48,360::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:48,360::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:48,361::__init__::232::IOProcess::(_processLogs) (7173) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:48,361::__init__::232::IOProcess::(_processLogs) (7173) Queuing response >Dummy-33::DEBUG::2014-08-11 15:30:48,546::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:48,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131524 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:50,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:50,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013399 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:52,554::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:52,556::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:30:52,561::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:30:52,650::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1163326-2e0f-4b33-8b34-e9fbae197226`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:52,652::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:52,652::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:52,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-115::INFO::2014-08-11 15:30:52,678::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:52,679::task::1191::Storage.TaskManager.Task::(prepare) Task=`c1163326-2e0f-4b33-8b34-e9fbae197226`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:30:52,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1163326-2e0f-4b33-8b34-e9fbae197226`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:52,680::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:52,680::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:52,681::task::993::Storage.TaskManager.Task::(_decref) Task=`c1163326-2e0f-4b33-8b34-e9fbae197226`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:30:52,692::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:30:52,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`1635086d-b7cd-426c-8252-c11567cfb6de`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:30:52,693::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:30:52,694::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`52dd42d4-41cf-4cb2-b272-a7024c729e1d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:30:52,694::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:30:52,695::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:30:52,695::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`52dd42d4-41cf-4cb2-b272-a7024c729e1d`::Granted request >Thread-34::DEBUG::2014-08-11 15:30:52,696::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1635086d-b7cd-426c-8252-c11567cfb6de`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:30:52,696::task::993::Storage.TaskManager.Task::(_decref) Task=`1635086d-b7cd-426c-8252-c11567cfb6de`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 15:30:52,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225846 s, 45.3 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 15:30:52,743::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:52,745::task::1191::Storage.TaskManager.Task::(prepare) Task=`1635086d-b7cd-426c-8252-c11567cfb6de`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:30:52,746::task::595::Storage.TaskManager.Task::(_updateState) Task=`1635086d-b7cd-426c-8252-c11567cfb6de`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:30:52,747::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:30:52,748::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:30:52,751::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:30:52,753::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:30:52,754::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:30:52,754::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:30:52,757::task::993::Storage.TaskManager.Task::(_decref) Task=`1635086d-b7cd-426c-8252-c11567cfb6de`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:30:54,596::task::595::Storage.TaskManager.Task::(_updateState) Task=`edaa07b9-1d09-497c-9632-b58fe7da3a70`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:30:54,596::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:30:54,597::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415614', 'lastCheck': '6.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:54,597::task::1191::Storage.TaskManager.Task::(prepare) Task=`edaa07b9-1d09-497c-9632-b58fe7da3a70`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415614', 'lastCheck': '6.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:30:54,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`edaa07b9-1d09-497c-9632-b58fe7da3a70`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:30:54,597::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:30:54,598::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:30:54,598::task::993::Storage.TaskManager.Task::(_decref) Task=`edaa07b9-1d09-497c-9632-b58fe7da3a70`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:30:54,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:30:54,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:54,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131816 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:30:56,822::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:56,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134945 s, 75.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:57,660::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:57,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:30:57,663::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:30:58,362::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:58,362::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:58,362::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:58,363::__init__::232::IOProcess::(_processLogs) (7174) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:58,363::__init__::232::IOProcess::(_processLogs) (7174) Queuing response >Thread-16::DEBUG::2014-08-11 15:30:58,364::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:30:58,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419732 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:30:58,384::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:58,385::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:58,385::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:58,385::__init__::232::IOProcess::(_processLogs) (7175) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:30:58,385::__init__::232::IOProcess::(_processLogs) (7175) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:58,387::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:58,387::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:58,388::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:58,388::__init__::232::IOProcess::(_processLogs) (7176) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:58,388::__init__::232::IOProcess::(_processLogs) (7176) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:58,389::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:58,389::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:58,390::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:58,390::__init__::232::IOProcess::(_processLogs) (7177) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:58,391::__init__::232::IOProcess::(_processLogs) (7177) Queuing response >Thread-14::DEBUG::2014-08-11 15:30:58,391::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:30:58,392::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:30:58,392::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:30:58,393::__init__::232::IOProcess::(_processLogs) (7178) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:30:58,393::__init__::232::IOProcess::(_processLogs) (7178) Queuing response >Dummy-33::DEBUG::2014-08-11 15:30:58,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:30:58,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134693 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:00,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:00,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135419 s, 75.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:02,671::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:02,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:02,751::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:31:02,775::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:31:02,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`b695f4de-361f-4d23-9bbd-d0ac5b9ee0f9`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:02,779::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:31:02,793::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:02,795::task::1191::Storage.TaskManager.Task::(prepare) Task=`b695f4de-361f-4d23-9bbd-d0ac5b9ee0f9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:02,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`b695f4de-361f-4d23-9bbd-d0ac5b9ee0f9`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:02,797::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:02,798::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:02,800::task::993::Storage.TaskManager.Task::(_decref) Task=`b695f4de-361f-4d23-9bbd-d0ac5b9ee0f9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:31:02,814::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:31:02,818::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7d29a17-6051-48ec-9162-8fb53eebd587`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:31:02,820::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:31:02,824::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`57448311-164e-49ac-b162-329f52dbdf25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:31:02,825::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:31:02,828::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:31:02,830::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`57448311-164e-49ac-b162-329f52dbdf25`::Granted request >Thread-34::DEBUG::2014-08-11 15:31:02,834::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b7d29a17-6051-48ec-9162-8fb53eebd587`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:31:02,836::task::993::Storage.TaskManager.Task::(_decref) Task=`b7d29a17-6051-48ec-9162-8fb53eebd587`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:31:02,895::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:02,896::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7d29a17-6051-48ec-9162-8fb53eebd587`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:02,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7d29a17-6051-48ec-9162-8fb53eebd587`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:31:02,899::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:31:02,901::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:31:02,903::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:31:02,905::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:31:02,907::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:31:02,908::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:31:02,911::task::993::Storage.TaskManager.Task::(_decref) Task=`b7d29a17-6051-48ec-9162-8fb53eebd587`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:31:03,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:03,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136337 s, 75.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:05,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:05,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013019 s, 78.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:07,111::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:07,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131779 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:07,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:07,762::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:07,848::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:31:08,394::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:08,394::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:08,395::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:08,395::__init__::232::IOProcess::(_processLogs) (7179) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:08,395::__init__::232::IOProcess::(_processLogs) (7179) Queuing response >Thread-16::DEBUG::2014-08-11 15:31:08,396::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:31:08,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420125 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:08,416::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:08,416::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:08,416::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:08,417::__init__::232::IOProcess::(_processLogs) (7180) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:08,417::__init__::232::IOProcess::(_processLogs) DEBUG|(7180) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:08,419::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:08,419::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:08,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:08,420::__init__::232::IOProcess::(_processLogs) (7181) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:08,420::__init__::232::IOProcess::(_processLogs) (7181) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:08,421::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:08,421::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:08,422::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:08,422::__init__::232::IOProcess::(_processLogs) (7182) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:08,423::__init__::232::IOProcess::(_processLogs) (7182) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:08,423::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:08,424::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:08,424::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:08,425::__init__::232::IOProcess::(_processLogs) (7183) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:08,425::__init__::232::IOProcess::(_processLogs) (7183) Queuing response >Dummy-33::DEBUG::2014-08-11 15:31:09,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:09,199::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132591 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:31:09,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebd14367-8197-4c92-89e9-1fc92f1de874`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:09,715::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:31:09,716::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420125', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:09,716::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebd14367-8197-4c92-89e9-1fc92f1de874`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420125', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:09,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebd14367-8197-4c92-89e9-1fc92f1de874`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:09,717::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:09,717::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:09,717::task::993::Storage.TaskManager.Task::(_decref) Task=`ebd14367-8197-4c92-89e9-1fc92f1de874`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:31:09,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:31:11,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:11,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131541 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:12,855::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:12,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:12,858::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:31:12,931::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:31:12,932::task::595::Storage.TaskManager.Task::(_updateState) Task=`f98a7630-d217-44c7-bfc8-393369a58e77`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:12,932::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:31:12,944::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:12,945::task::1191::Storage.TaskManager.Task::(prepare) Task=`f98a7630-d217-44c7-bfc8-393369a58e77`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:12,945::task::595::Storage.TaskManager.Task::(_updateState) Task=`f98a7630-d217-44c7-bfc8-393369a58e77`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:12,945::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:12,945::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:12,946::task::993::Storage.TaskManager.Task::(_decref) Task=`f98a7630-d217-44c7-bfc8-393369a58e77`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:31:12,957::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:31:12,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`e224afad-2385-4e71-90a1-7aafe55ff61f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:31:12,958::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:31:12,958::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`41314dc8-9081-4a5d-851e-aca7aaa508d2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:31:12,959::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:31:12,959::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:31:12,959::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`41314dc8-9081-4a5d-851e-aca7aaa508d2`::Granted request >Thread-34::DEBUG::2014-08-11 15:31:12,960::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e224afad-2385-4e71-90a1-7aafe55ff61f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:31:12,960::task::993::Storage.TaskManager.Task::(_decref) Task=`e224afad-2385-4e71-90a1-7aafe55ff61f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:31:12,980::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:12,981::task::1191::Storage.TaskManager.Task::(prepare) Task=`e224afad-2385-4e71-90a1-7aafe55ff61f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:12,981::task::595::Storage.TaskManager.Task::(_updateState) Task=`e224afad-2385-4e71-90a1-7aafe55ff61f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:31:12,981::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:31:12,981::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:31:12,982::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:31:12,982::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:31:12,982::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:31:12,982::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:31:12,983::task::993::Storage.TaskManager.Task::(_decref) Task=`e224afad-2385-4e71-90a1-7aafe55ff61f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:31:13,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:13,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132832 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:15,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:15,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132583 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:17,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:17,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133748 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:17,864::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:17,866::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:17,948::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:31:18,426::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:18,426::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:18,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:18,427::__init__::232::IOProcess::(_processLogs) (7184) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:18,427::__init__::232::IOProcess::(_processLogs) (7184) Queuing response >Thread-16::DEBUG::2014-08-11 15:31:18,427::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:31:18,446::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416777 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:18,447::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:18,447::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:18,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:18,448::__init__::232::IOProcess::(_processLogs) (7185) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:18,448::__init__::232::IOProcess::(_processLogs) (7185) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:18,449::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:18,450::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:18,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:18,451::__init__::232::IOProcess::(_processLogs) (7186) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:18,451::__init__::232::IOProcess::(_processLogs) (7186) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:18,451::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:18,452::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:18,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:18,453::__init__::232::IOProcess::(_processLogs) (7187) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:18,453::__init__::232::IOProcess::(_processLogs) (7187) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:18,454::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:18,454::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:18,454::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:18,455::__init__::232::IOProcess::(_processLogs) (7188) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:18,456::__init__::232::IOProcess::(_processLogs) (7188) Queuing response >Dummy-33::DEBUG::2014-08-11 15:31:19,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:19,474::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132729 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:21,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:21,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132891 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:22,955::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:22,957::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:31:22,994::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:31:23,050::task::595::Storage.TaskManager.Task::(_updateState) Task=`bebd1772-16fa-448a-8005-f3368d33023c`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:23,052::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:31:23,052::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:31:23,067::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:23,067::task::1191::Storage.TaskManager.Task::(prepare) Task=`bebd1772-16fa-448a-8005-f3368d33023c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:23,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`bebd1772-16fa-448a-8005-f3368d33023c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:23,068::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:23,068::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:23,068::task::993::Storage.TaskManager.Task::(_decref) Task=`bebd1772-16fa-448a-8005-f3368d33023c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:31:23,079::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:31:23,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`aeed86c7-dd00-47fc-907d-545a710f4312`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:31:23,080::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:31:23,081::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8f760c7f-6ce2-4385-bc5f-a594c2e3ba4c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:31:23,081::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:31:23,081::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:31:23,081::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8f760c7f-6ce2-4385-bc5f-a594c2e3ba4c`::Granted request >Thread-34::DEBUG::2014-08-11 15:31:23,082::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`aeed86c7-dd00-47fc-907d-545a710f4312`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:31:23,082::task::993::Storage.TaskManager.Task::(_decref) Task=`aeed86c7-dd00-47fc-907d-545a710f4312`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:31:23,102::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:23,102::task::1191::Storage.TaskManager.Task::(prepare) Task=`aeed86c7-dd00-47fc-907d-545a710f4312`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:23,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`aeed86c7-dd00-47fc-907d-545a710f4312`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:31:23,103::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:31:23,103::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:31:23,104::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:31:23,104::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:31:23,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:31:23,104::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:31:23,105::task::993::Storage.TaskManager.Task::(_decref) Task=`aeed86c7-dd00-47fc-907d-545a710f4312`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:31:23,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:23,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132456 s, 77.3 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:31:23,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`c928f654-61ca-419e-ab5e-e7bd02b6a0d5`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:31:23,821::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:31:23,823::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:23,823::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:23,823::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:23,823::__init__::232::IOProcess::(_processLogs) (7189) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:31:23,824::__init__::232::IOProcess::(_processLogs) DEBUG|(7189) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:23,825::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:23,826::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:31:23,826::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:31:23,827::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:31:23,828::task::1191::Storage.TaskManager.Task::(prepare) Task=`c928f654-61ca-419e-ab5e-e7bd02b6a0d5`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:31:23,828::__init__::232::IOProcess::(_processLogs) (7190) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:31:23,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`c928f654-61ca-419e-ab5e-e7bd02b6a0d5`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:31:23,830::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:31:23,830::__init__::232::IOProcess::(_processLogs) (7190) Queuing response >Thread-122::DEBUG::2014-08-11 15:31:23,831::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:31:23,832::task::993::Storage.TaskManager.Task::(_decref) Task=`c928f654-61ca-419e-ab5e-e7bd02b6a0d5`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:31:24,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed449820-b156-45d2-8a90-babba87d2a97`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:24,831::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:31:24,831::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416777', 'lastCheck': '6.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:24,832::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed449820-b156-45d2-8a90-babba87d2a97`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416777', 'lastCheck': '6.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:24,832::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed449820-b156-45d2-8a90-babba87d2a97`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:24,832::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:24,832::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:24,832::task::993::Storage.TaskManager.Task::(_decref) Task=`ed449820-b156-45d2-8a90-babba87d2a97`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:31:24,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:31:25,609::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:25,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131738 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:27,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:27,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135273 s, 75.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:28,061::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:28,062::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:28,148::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:31:28,456::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:28,457::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:28,457::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:28,457::__init__::232::IOProcess::(_processLogs) (7191) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:28,457::__init__::232::IOProcess::(_processLogs) (7191) Queuing response >Thread-16::DEBUG::2014-08-11 15:31:28,458::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:31:28,477::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416484 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:28,478::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:28,478::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:28,479::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:28,479::__init__::232::IOProcess::(_processLogs) (7192) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:28,479::__init__::232::IOProcess::(_processLogs) (7192) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:28,480::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:28,481::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:28,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:28,481::__init__::232::IOProcess::(_processLogs) (7193) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:28,482::__init__::232::IOProcess::(_processLogs) DEBUG|(7193) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:28,483::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:28,483::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:28,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:28,484::__init__::232::IOProcess::(_processLogs) (7194) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:28,484::__init__::232::IOProcess::(_processLogs) (7194) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:28,485::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:28,485::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:28,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:28,486::__init__::232::IOProcess::(_processLogs) (7195) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:28,486::__init__::232::IOProcess::(_processLogs) DEBUG|(7195) Queuing response >Dummy-33::DEBUG::2014-08-11 15:31:29,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:29,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132341 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:31,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:31,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133647 s, 76.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:31:33,120::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:31:33,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f915804-a4b6-44c8-8ef2-8e8cc908ee3e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:33,148::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:33,155::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:33,158::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:33,160::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:31:33,162::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:33,162::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f915804-a4b6-44c8-8ef2-8e8cc908ee3e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:33,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f915804-a4b6-44c8-8ef2-8e8cc908ee3e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:33,162::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:33,163::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:33,163::task::993::Storage.TaskManager.Task::(_decref) Task=`5f915804-a4b6-44c8-8ef2-8e8cc908ee3e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:31:33,173::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:31:33,174::task::595::Storage.TaskManager.Task::(_updateState) Task=`2707fa0a-715c-415e-bbbd-0aee787dad09`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:31:33,174::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:31:33,175::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`293ff161-5919-4d84-b91c-d66bc5445206`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:31:33,175::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:31:33,175::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:31:33,176::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`293ff161-5919-4d84-b91c-d66bc5445206`::Granted request >Thread-34::DEBUG::2014-08-11 15:31:33,176::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2707fa0a-715c-415e-bbbd-0aee787dad09`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:31:33,176::task::993::Storage.TaskManager.Task::(_decref) Task=`2707fa0a-715c-415e-bbbd-0aee787dad09`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:31:33,196::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:33,196::task::1191::Storage.TaskManager.Task::(prepare) Task=`2707fa0a-715c-415e-bbbd-0aee787dad09`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:33,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`2707fa0a-715c-415e-bbbd-0aee787dad09`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:31:33,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:31:33,197::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:31:33,197::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:31:33,198::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:31:33,198::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:31:33,198::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:31:33,199::task::993::Storage.TaskManager.Task::(_decref) Task=`2707fa0a-715c-415e-bbbd-0aee787dad09`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:31:33,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:33,874::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0248919 s, 41.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:35,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:35,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131478 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:37,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:37,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133708 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:38,166::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:38,168::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:38,248::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:31:38,488::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:38,488::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:38,488::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:38,489::__init__::232::IOProcess::(_processLogs) (7196) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:38,490::__init__::232::IOProcess::(_processLogs) DEBUG|(7196) Queuing response >Thread-16::DEBUG::2014-08-11 15:31:38,491::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:31:38,509::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417643 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:38,510::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:38,510::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:38,511::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:38,511::__init__::232::IOProcess::(_processLogs) (7197) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:38,512::__init__::232::IOProcess::(_processLogs) (7197) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:38,513::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:38,513::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:38,514::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:38,514::__init__::232::IOProcess::(_processLogs) (7198) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:38,515::__init__::232::IOProcess::(_processLogs) (7198) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:38,515::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:38,515::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:38,515::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:38,516::__init__::232::IOProcess::(_processLogs) (7199) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:38,516::__init__::232::IOProcess::(_processLogs) DEBUG|(7199) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:38,517::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:38,518::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:38,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:38,519::__init__::232::IOProcess::(_processLogs) (7200) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:38,519::__init__::232::IOProcess::(_processLogs) (7200) Queuing response >Thread-115::DEBUG::2014-08-11 15:31:39,941::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b4beb2-473e-4bf8-aa90-7f5362dede4e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:39,941::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:31:39,941::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417643', 'lastCheck': '1.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:39,942::task::1191::Storage.TaskManager.Task::(prepare) Task=`68b4beb2-473e-4bf8-aa90-7f5362dede4e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000417643', 'lastCheck': '1.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:39,942::task::595::Storage.TaskManager.Task::(_updateState) Task=`68b4beb2-473e-4bf8-aa90-7f5362dede4e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:39,942::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:39,942::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:39,942::task::993::Storage.TaskManager.Task::(_decref) Task=`68b4beb2-473e-4bf8-aa90-7f5362dede4e`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:31:39,962::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:31:40,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:40,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133508 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:42,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:42,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132804 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:31:43,211::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:31:43,212::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc1711ac-b096-44ef-87d4-68a17fc4ce05`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:43,212::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:43,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:43,257::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:43,258::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:31:43,260::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:43,260::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc1711ac-b096-44ef-87d4-68a17fc4ce05`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:43,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc1711ac-b096-44ef-87d4-68a17fc4ce05`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:43,261::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:43,261::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:43,261::task::993::Storage.TaskManager.Task::(_decref) Task=`cc1711ac-b096-44ef-87d4-68a17fc4ce05`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:31:43,274::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:31:43,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dc19951-7d2f-4992-8da3-e2875b3a9412`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:31:43,275::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:31:43,276::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3bed4409-fed3-4ed0-9520-6b46fb38b3a0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:31:43,276::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:31:43,276::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:31:43,277::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3bed4409-fed3-4ed0-9520-6b46fb38b3a0`::Granted request >Thread-34::DEBUG::2014-08-11 15:31:43,277::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0dc19951-7d2f-4992-8da3-e2875b3a9412`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:31:43,277::task::993::Storage.TaskManager.Task::(_decref) Task=`0dc19951-7d2f-4992-8da3-e2875b3a9412`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:31:43,297::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:43,298::task::1191::Storage.TaskManager.Task::(prepare) Task=`0dc19951-7d2f-4992-8da3-e2875b3a9412`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:43,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dc19951-7d2f-4992-8da3-e2875b3a9412`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:31:43,298::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:31:43,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:31:43,299::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:31:43,299::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:31:43,299::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:31:43,299::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:31:43,300::task::993::Storage.TaskManager.Task::(_decref) Task=`0dc19951-7d2f-4992-8da3-e2875b3a9412`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:31:44,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:44,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132321 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:46,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:46,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132583 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:48,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:48,263::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:48,264::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:48,265::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:31:48,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0170377 s, 60.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:48,520::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:48,520::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:48,521::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:48,521::__init__::232::IOProcess::(_processLogs) (7201) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:48,521::__init__::232::IOProcess::(_processLogs) (7201) Queuing response >Thread-16::DEBUG::2014-08-11 15:31:48,522::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:31:48,542::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415944 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:48,543::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:48,543::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:48,543::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:48,544::__init__::232::IOProcess::(_processLogs) (7202) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:48,544::__init__::232::IOProcess::(_processLogs) (7202) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:48,545::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:48,546::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:48,546::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:48,547::__init__::232::IOProcess::(_processLogs) (7203) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:48,547::__init__::232::IOProcess::(_processLogs) (7203) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:48,547::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:48,548::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:48,548::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:48,549::__init__::232::IOProcess::(_processLogs) (7204) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:48,549::__init__::232::IOProcess::(_processLogs) (7204) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:48,549::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:48,550::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:48,550::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:48,551::__init__::232::IOProcess::(_processLogs) (7205) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:48,552::__init__::232::IOProcess::(_processLogs) (7205) Queuing response >Dummy-33::DEBUG::2014-08-11 15:31:50,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:50,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134321 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:31:52,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:52,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131392 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:53,272::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:53,273::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:31:53,312::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:31:53,350::task::595::Storage.TaskManager.Task::(_updateState) Task=`82c3d64b-5d74-4044-be84-3331a830b459`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:53,352::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:31:53,352::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:31:53,366::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:53,367::task::1191::Storage.TaskManager.Task::(prepare) Task=`82c3d64b-5d74-4044-be84-3331a830b459`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:31:53,367::task::595::Storage.TaskManager.Task::(_updateState) Task=`82c3d64b-5d74-4044-be84-3331a830b459`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:53,367::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:53,367::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:53,368::task::993::Storage.TaskManager.Task::(_decref) Task=`82c3d64b-5d74-4044-be84-3331a830b459`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:31:53,378::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:31:53,379::task::595::Storage.TaskManager.Task::(_updateState) Task=`eec053cc-6c1b-4ff5-9965-4125f28e5a8c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:31:53,379::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:31:53,380::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9d708b3d-5acd-4b02-b890-c128ed5514c4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:31:53,380::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:31:53,381::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:31:53,381::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9d708b3d-5acd-4b02-b890-c128ed5514c4`::Granted request >Thread-34::DEBUG::2014-08-11 15:31:53,381::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`eec053cc-6c1b-4ff5-9965-4125f28e5a8c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:31:53,382::task::993::Storage.TaskManager.Task::(_decref) Task=`eec053cc-6c1b-4ff5-9965-4125f28e5a8c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:31:53,401::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:53,402::task::1191::Storage.TaskManager.Task::(prepare) Task=`eec053cc-6c1b-4ff5-9965-4125f28e5a8c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:31:53,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`eec053cc-6c1b-4ff5-9965-4125f28e5a8c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:31:53,402::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:31:53,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:31:53,403::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:31:53,403::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:31:53,403::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:31:53,404::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:31:53,404::task::993::Storage.TaskManager.Task::(_decref) Task=`eec053cc-6c1b-4ff5-9965-4125f28e5a8c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:31:54,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:54,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013137 s, 77.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:31:55,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e5a0dc0-0342-4088-bd31-a4e849eb885d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:31:55,052::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:31:55,052::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415944', 'lastCheck': '6.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:55,052::task::1191::Storage.TaskManager.Task::(prepare) Task=`2e5a0dc0-0342-4088-bd31-a4e849eb885d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415944', 'lastCheck': '6.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:31:55,053::task::595::Storage.TaskManager.Task::(_updateState) Task=`2e5a0dc0-0342-4088-bd31-a4e849eb885d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:31:55,053::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:31:55,053::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:31:55,053::task::993::Storage.TaskManager.Task::(_decref) Task=`2e5a0dc0-0342-4088-bd31-a4e849eb885d`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:31:55,076::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:31:56,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:56,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132476 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:58,356::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:58,357::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:31:58,449::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:31:58,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:31:58,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013325 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:58,553::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:58,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:58,556::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:58,557::__init__::232::IOProcess::(_processLogs) (7206) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:58,559::__init__::232::IOProcess::(_processLogs) DEBUG|(7206) Queuing response >Thread-16::DEBUG::2014-08-11 15:31:58,561::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:31:58,581::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00049604 s, 917 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:31:58,584::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:58,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:58,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:58,588::__init__::232::IOProcess::(_processLogs) (7207) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:31:58,589::__init__::232::IOProcess::(_processLogs) DEBUG|(7207) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:58,592::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:58,593::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:58,594::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:58,595::__init__::232::IOProcess::(_processLogs) (7208) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:58,597::__init__::232::IOProcess::(_processLogs) DEBUG|(7208) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:58,599::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:58,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:58,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:58,602::__init__::232::IOProcess::(_processLogs) (7209) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:58,604::__init__::232::IOProcess::(_processLogs) DEBUG|(7209) Queuing response >Thread-14::DEBUG::2014-08-11 15:31:58,606::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:31:58,608::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:31:58,609::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:31:58,610::__init__::232::IOProcess::(_processLogs) (7210) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:31:58,612::__init__::232::IOProcess::(_processLogs) (7210) Queuing response >Dummy-33::DEBUG::2014-08-11 15:32:00,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:00,662::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133785 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:02,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:02,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0271956 s, 37.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:32:03,417::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-115::DEBUG::2014-08-11 15:32:03,418::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb120abe-36ed-4bd8-9971-923aba20bdd2`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:03,448::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:03,458::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:03,460::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:03,461::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:32:03,461::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:03,463::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb120abe-36ed-4bd8-9971-923aba20bdd2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:03,464::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb120abe-36ed-4bd8-9971-923aba20bdd2`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:03,465::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:03,465::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:03,466::task::993::Storage.TaskManager.Task::(_decref) Task=`eb120abe-36ed-4bd8-9971-923aba20bdd2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:32:03,478::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 15:32:03,479::task::595::Storage.TaskManager.Task::(_updateState) Task=`36d6991c-cbeb-4640-9f58-1af127484d3f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:32:03,479::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:32:03,480::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`95ab28bc-257f-48d7-9b0e-a69ab254985d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:32:03,480::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:32:03,481::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:32:03,481::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`95ab28bc-257f-48d7-9b0e-a69ab254985d`::Granted request >Thread-34::DEBUG::2014-08-11 15:32:03,481::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`36d6991c-cbeb-4640-9f58-1af127484d3f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:32:03,482::task::993::Storage.TaskManager.Task::(_decref) Task=`36d6991c-cbeb-4640-9f58-1af127484d3f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:32:03,501::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:03,502::task::1191::Storage.TaskManager.Task::(prepare) Task=`36d6991c-cbeb-4640-9f58-1af127484d3f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:03,502::task::595::Storage.TaskManager.Task::(_updateState) Task=`36d6991c-cbeb-4640-9f58-1af127484d3f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:32:03,502::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:32:03,502::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:32:03,503::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:32:03,503::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:32:03,503::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:32:03,504::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:32:03,504::task::993::Storage.TaskManager.Task::(_decref) Task=`36d6991c-cbeb-4640-9f58-1af127484d3f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:32:04,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:04,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013319 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:06,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:06,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132193 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:08,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:08,473::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:08,549::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:08,609::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:08,609::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:08,609::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:08,610::__init__::232::IOProcess::(_processLogs) DEBUG|(7211) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:32:08,611::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:32:08,612::__init__::232::IOProcess::(_processLogs) (7211) Queuing response >Thread-16::DEBUG::2014-08-11 15:32:08,631::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000433805 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:08,632::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:08,633::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:08,633::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:08,633::__init__::232::IOProcess::(_processLogs) (7212) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:08,634::__init__::232::IOProcess::(_processLogs) DEBUG|(7212) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:08,635::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:08,636::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:08,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:08,637::__init__::232::IOProcess::(_processLogs) (7213) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:08,637::__init__::232::IOProcess::(_processLogs) (7213) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:08,637::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:08,638::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:08,639::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:08,639::__init__::232::IOProcess::(_processLogs) (7214) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:08,639::__init__::232::IOProcess::(_processLogs) (7214) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:08,640::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:08,640::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:08,641::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:08,642::__init__::232::IOProcess::(_processLogs) (7215) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:08,642::__init__::232::IOProcess::(_processLogs) (7215) Queuing response >Dummy-33::DEBUG::2014-08-11 15:32:09,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:09,054::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132516 s, 77.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:32:10,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`7dca4911-fd95-4f0b-bcfa-132f3d0d2bf0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:10,165::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:32:10,165::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000433805', 'lastCheck': '1.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:10,165::task::1191::Storage.TaskManager.Task::(prepare) Task=`7dca4911-fd95-4f0b-bcfa-132f3d0d2bf0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000433805', 'lastCheck': '1.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:10,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`7dca4911-fd95-4f0b-bcfa-132f3d0d2bf0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:10,166::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:10,166::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:10,166::task::993::Storage.TaskManager.Task::(_decref) Task=`7dca4911-fd95-4f0b-bcfa-132f3d0d2bf0`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:32:10,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:32:11,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:11,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013255 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:13,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:13,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133656 s, 76.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:32:13,516::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:32:13,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`34ee8d7b-ce93-4b32-95cf-e5ed58e87a0c`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:13,556::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:32:13,649::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:13,652::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:13,653::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:32:13,665::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:13,666::task::1191::Storage.TaskManager.Task::(prepare) Task=`34ee8d7b-ce93-4b32-95cf-e5ed58e87a0c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:13,666::task::595::Storage.TaskManager.Task::(_updateState) Task=`34ee8d7b-ce93-4b32-95cf-e5ed58e87a0c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:13,666::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:13,666::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:13,667::task::993::Storage.TaskManager.Task::(_decref) Task=`34ee8d7b-ce93-4b32-95cf-e5ed58e87a0c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:32:13,678::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:32:13,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd41d066-676a-4ae6-a84f-1fba721fbd70`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:32:13,679::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:32:13,680::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`21253359-f754-469f-88ec-e4a5162ba3a1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:32:13,680::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:32:13,680::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:32:13,681::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`21253359-f754-469f-88ec-e4a5162ba3a1`::Granted request >Thread-34::DEBUG::2014-08-11 15:32:13,681::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dd41d066-676a-4ae6-a84f-1fba721fbd70`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:32:13,681::task::993::Storage.TaskManager.Task::(_decref) Task=`dd41d066-676a-4ae6-a84f-1fba721fbd70`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:32:13,701::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:13,702::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd41d066-676a-4ae6-a84f-1fba721fbd70`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:13,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd41d066-676a-4ae6-a84f-1fba721fbd70`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:32:13,702::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:32:13,702::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:32:13,703::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:32:13,703::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:32:13,703::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:32:13,703::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:32:13,704::task::993::Storage.TaskManager.Task::(_decref) Task=`dd41d066-676a-4ae6-a84f-1fba721fbd70`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:32:15,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:15,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132273 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:17,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:17,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01331 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:18,642::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:18,643::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:18,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:18,649::__init__::232::IOProcess::(_processLogs) (7216) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:18,649::__init__::232::IOProcess::(_processLogs) (7216) Queuing response >Thread-16::DEBUG::2014-08-11 15:32:18,650::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:18,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:18,670::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:18,671::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:32:18,673::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418918 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:18,674::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:18,674::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:18,674::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:18,675::__init__::232::IOProcess::(_processLogs) (7217) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:18,675::__init__::232::IOProcess::(_processLogs) DEBUG|(7217) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:18,677::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:18,677::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:18,678::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:18,678::__init__::232::IOProcess::(_processLogs) (7218) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:18,678::__init__::232::IOProcess::(_processLogs) (7218) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:18,679::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:18,679::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:18,680::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:18,680::__init__::232::IOProcess::(_processLogs) (7219) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:18,680::__init__::232::IOProcess::(_processLogs) (7219) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:18,681::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:18,681::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:18,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:18,683::__init__::232::IOProcess::(_processLogs) (7220) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:18,683::__init__::232::IOProcess::(_processLogs) (7220) Queuing response >Dummy-33::DEBUG::2014-08-11 15:32:19,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:19,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013367 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:21,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:21,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133369 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:23,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:23,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132023 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:23,678::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:23,679::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:32:23,717::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:32:23,751::task::595::Storage.TaskManager.Task::(_updateState) Task=`8acd486a-0e8b-4ef2-93f6-1d7cce47a1c0`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:23,752::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:23,754::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:32:23,767::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:23,767::task::1191::Storage.TaskManager.Task::(prepare) Task=`8acd486a-0e8b-4ef2-93f6-1d7cce47a1c0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:23,767::task::595::Storage.TaskManager.Task::(_updateState) Task=`8acd486a-0e8b-4ef2-93f6-1d7cce47a1c0`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:23,768::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:23,768::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:23,768::task::993::Storage.TaskManager.Task::(_decref) Task=`8acd486a-0e8b-4ef2-93f6-1d7cce47a1c0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:32:23,779::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:32:23,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`061ad107-689b-46ee-82c0-18892caa7c54`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:32:23,780::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:32:23,781::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1492c726-7902-4522-bc4b-33668b4a0d8b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:32:23,781::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:32:23,782::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:32:23,782::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1492c726-7902-4522-bc4b-33668b4a0d8b`::Granted request >Thread-34::DEBUG::2014-08-11 15:32:23,783::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`061ad107-689b-46ee-82c0-18892caa7c54`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:32:23,783::task::993::Storage.TaskManager.Task::(_decref) Task=`061ad107-689b-46ee-82c0-18892caa7c54`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:32:23,803::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:23,803::task::1191::Storage.TaskManager.Task::(prepare) Task=`061ad107-689b-46ee-82c0-18892caa7c54`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:23,803::task::595::Storage.TaskManager.Task::(_updateState) Task=`061ad107-689b-46ee-82c0-18892caa7c54`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:32:23,804::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:32:23,804::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:32:23,804::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:32:23,804::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:32:23,805::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:32:23,805::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:32:23,805::task::993::Storage.TaskManager.Task::(_decref) Task=`061ad107-689b-46ee-82c0-18892caa7c54`::ref 0 aborting False >Thread-122::DEBUG::2014-08-11 15:32:23,936::task::595::Storage.TaskManager.Task::(_updateState) Task=`723ca88e-bc65-413e-acfc-e12a0feaf8b0`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:32:23,937::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:32:23,938::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:23,938::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:23,938::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:23,939::__init__::232::IOProcess::(_processLogs) (7221) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:32:23,940::__init__::232::IOProcess::(_processLogs) DEBUG|(7221) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:23,941::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:23,941::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:32:23,942::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:32:23,942::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:32:23,943::task::1191::Storage.TaskManager.Task::(prepare) Task=`723ca88e-bc65-413e-acfc-e12a0feaf8b0`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:32:23,944::__init__::232::IOProcess::(_processLogs) (7222) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:32:23,944::task::595::Storage.TaskManager.Task::(_updateState) Task=`723ca88e-bc65-413e-acfc-e12a0feaf8b0`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:32:23,945::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:32:23,946::__init__::232::IOProcess::(_processLogs) (7222) Queuing response >Thread-122::DEBUG::2014-08-11 15:32:23,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:32:23,947::task::993::Storage.TaskManager.Task::(_decref) Task=`723ca88e-bc65-413e-acfc-e12a0feaf8b0`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:32:25,277::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9bc1fe1-1c73-4284-9dd4-fb6d451e405c`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:25,278::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:32:25,278::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418918', 'lastCheck': '6.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:25,278::task::1191::Storage.TaskManager.Task::(prepare) Task=`f9bc1fe1-1c73-4284-9dd4-fb6d451e405c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418918', 'lastCheck': '6.6', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:25,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`f9bc1fe1-1c73-4284-9dd4-fb6d451e405c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:25,279::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:25,279::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:25,279::task::993::Storage.TaskManager.Task::(_decref) Task=`f9bc1fe1-1c73-4284-9dd4-fb6d451e405c`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:32:25,298::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:32:25,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:25,497::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132818 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:27,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:27,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131802 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:28,684::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:28,685::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:32:28,686::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:32:28,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:28,768::__init__::232::IOProcess::(_processLogs) (7223) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:28,770::__init__::232::IOProcess::(_processLogs) (7223) Queuing response >Thread-16::DEBUG::2014-08-11 15:32:28,771::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000831678 s, 547 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:28,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:28,773::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:28,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:28,776::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:28,778::__init__::232::IOProcess::(_processLogs) (7224) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:28,778::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:28,780::__init__::232::IOProcess::(_processLogs) DEBUG|(7224) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:28,782::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:28,784::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:28,785::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:28,786::__init__::232::IOProcess::(_processLogs) (7225) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:28,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:28,788::__init__::232::IOProcess::(_processLogs) DEBUG|(7225) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:28,790::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:28,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:28,791::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:28,791::__init__::232::IOProcess::(_processLogs) (7226) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:28,792::__init__::232::IOProcess::(_processLogs) (7226) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:28,793::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:28,793::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:28,793::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:28,793::__init__::232::IOProcess::(_processLogs) (7227) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:28,794::__init__::232::IOProcess::(_processLogs) DEBUG|(7227) Queuing response >Dummy-33::DEBUG::2014-08-11 15:32:29,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:29,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133232 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:31,631::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:31,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134464 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:33,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:33,720::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133748 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:33,797::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:33,798::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:33,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:32:33,822::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:32:33,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ebf0a22-5489-419e-badb-de30a0157a64`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:33,823::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:32:33,837::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:33,837::task::1191::Storage.TaskManager.Task::(prepare) Task=`6ebf0a22-5489-419e-badb-de30a0157a64`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:33,837::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ebf0a22-5489-419e-badb-de30a0157a64`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:33,837::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:33,838::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:33,838::task::993::Storage.TaskManager.Task::(_decref) Task=`6ebf0a22-5489-419e-badb-de30a0157a64`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:32:33,848::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:32:33,848::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef55cc31-71f9-450a-adc7-2e2769555681`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:32:33,849::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:32:33,850::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8bed88a0-2cd1-432f-b355-afcbac0012d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:32:33,850::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:32:33,850::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:32:33,851::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8bed88a0-2cd1-432f-b355-afcbac0012d4`::Granted request >Thread-34::DEBUG::2014-08-11 15:32:33,851::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ef55cc31-71f9-450a-adc7-2e2769555681`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:32:33,851::task::993::Storage.TaskManager.Task::(_decref) Task=`ef55cc31-71f9-450a-adc7-2e2769555681`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:32:33,871::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:33,871::task::1191::Storage.TaskManager.Task::(prepare) Task=`ef55cc31-71f9-450a-adc7-2e2769555681`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:33,872::task::595::Storage.TaskManager.Task::(_updateState) Task=`ef55cc31-71f9-450a-adc7-2e2769555681`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:32:33,872::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:32:33,872::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:32:33,872::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:32:33,873::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:32:33,873::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:32:33,873::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:32:33,873::task::993::Storage.TaskManager.Task::(_decref) Task=`ef55cc31-71f9-450a-adc7-2e2769555681`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:32:35,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:35,803::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134221 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:37,825::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:37,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132612 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:38,796::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:38,801::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:38,850::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:32:38,852::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:32:38,852::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:38,871::__init__::232::IOProcess::(_processLogs) (7228) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:38,873::__init__::232::IOProcess::(_processLogs) (7228) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:38,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:32:38,875::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00105856 s, 430 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:38,878::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:38,879::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:38,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:38,881::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:38,882::__init__::232::IOProcess::(_processLogs) (7229) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:38,885::__init__::232::IOProcess::(_processLogs) DEBUG|(7229) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:38,886::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:38,886::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:38,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:38,886::__init__::232::IOProcess::(_processLogs) (7230) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:38,887::__init__::232::IOProcess::(_processLogs) (7230) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:38,887::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:38,888::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:38,888::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:38,888::__init__::232::IOProcess::(_processLogs) (7231) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:38,889::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7231) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:38,890::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:38,890::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:38,891::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:38,892::__init__::232::IOProcess::(_processLogs) (7232) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:38,892::__init__::232::IOProcess::(_processLogs) (7232) Queuing response >Dummy-33::DEBUG::2014-08-11 15:32:39,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:39,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133543 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:32:40,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`569d37b0-1140-4c9b-b85c-b5ffe8a5faa4`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:40,414::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:32:40,415::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00105856', 'lastCheck': '1.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:40,415::task::1191::Storage.TaskManager.Task::(prepare) Task=`569d37b0-1140-4c9b-b85c-b5ffe8a5faa4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00105856', 'lastCheck': '1.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:40,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`569d37b0-1140-4c9b-b85c-b5ffe8a5faa4`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:40,415::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:40,416::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:40,416::task::993::Storage.TaskManager.Task::(_decref) Task=`569d37b0-1140-4c9b-b85c-b5ffe8a5faa4`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:32:40,434::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:32:41,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:41,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132582 s, 77.2 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:32:43,886::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:32:43,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`2da8a834-90df-4804-9355-d97c069b3abc`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:43,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:32:43,949::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:43,951::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:43,953::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:32:43,966::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:43,966::task::1191::Storage.TaskManager.Task::(prepare) Task=`2da8a834-90df-4804-9355-d97c069b3abc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:43,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`2da8a834-90df-4804-9355-d97c069b3abc`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:43,967::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:43,967::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:43,967::task::993::Storage.TaskManager.Task::(_decref) Task=`2da8a834-90df-4804-9355-d97c069b3abc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:32:43,982::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:32:43,982::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f2688e6-6532-47c2-b5e1-78862782b1d2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:32:43,983::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:32:43,983::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`46496abf-38f6-4f4b-93c4-5b3bc357b0e4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:32:43,984::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:32:43,984::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:32:43,984::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`46496abf-38f6-4f4b-93c4-5b3bc357b0e4`::Granted request >Thread-34::DEBUG::2014-08-11 15:32:43,985::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3f2688e6-6532-47c2-b5e1-78862782b1d2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:32:43,985::task::993::Storage.TaskManager.Task::(_decref) Task=`3f2688e6-6532-47c2-b5e1-78862782b1d2`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 15:32:43,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:44,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132146 s, 77.5 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 15:32:44,038::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:44,039::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f2688e6-6532-47c2-b5e1-78862782b1d2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:44,040::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f2688e6-6532-47c2-b5e1-78862782b1d2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:32:44,041::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:32:44,042::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:32:44,044::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:32:44,046::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:32:44,047::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:32:44,048::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:32:44,050::task::993::Storage.TaskManager.Task::(_decref) Task=`3f2688e6-6532-47c2-b5e1-78862782b1d2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:32:46,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:46,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.018182 s, 56.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:48,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:48,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132567 s, 77.2 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:32:48,892::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:32:48,894::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:48,949::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:48,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:48,954::__init__::232::IOProcess::(_processLogs) (7233) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:32:48,954::__init__::232::IOProcess::(_processLogs) (7233) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:48,962::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:32:48,962::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:32:48,965::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:48,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:48,968::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:48,968::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:48,970::__init__::232::IOProcess::(_processLogs) (7234) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:32:48,972::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7234) Queuing response >Thread-16::DEBUG::2014-08-11 15:32:48,973::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:48,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:48,977::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:48,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:48,978::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:48,978::__init__::232::IOProcess::(_processLogs) (7235) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:32:48,979::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7235) Queuing response >Thread-16::DEBUG::2014-08-11 15:32:48,979::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:32:48,981::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:48,981::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:48,982::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:48,982::__init__::232::IOProcess::(_processLogs) (7236) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:48,982::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7236) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:48,983::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:48,984::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:48,984::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:48,984::__init__::232::IOProcess::(_processLogs) (7237) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:32:48,985::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 15:32:48,985::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7237) Queuing response >Thread-16::INFO::2014-08-11 15:32:48,986::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:32:48,986::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:32:48,987::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:48,987::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:48,987::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:48,988::__init__::232::IOProcess::(_processLogs) (7238) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:48,988::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7238) Queuing response >Thread-16::DEBUG::2014-08-11 15:32:48,990::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:32:49,008::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418067 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:49,010::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:49,010::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:49,010::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:49,010::__init__::232::IOProcess::(_processLogs) (7239) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:49,011::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7239) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:49,012::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:49,012::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:49,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:49,013::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7240) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:49,013::__init__::232::IOProcess::(_processLogs) (7240) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:49,014::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:49,014::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:49,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:49,015::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7241) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:49,015::__init__::232::IOProcess::(_processLogs) (7241) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:49,016::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:49,016::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:49,016::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:49,016::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7242) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:49,017::__init__::232::IOProcess::(_processLogs) (7242) Queuing response >Dummy-33::DEBUG::2014-08-11 15:32:50,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:50,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133388 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:52,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:52,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132762 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:53,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:53,985::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:54,050::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:32:54,065::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:32:54,066::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ba3d803-3df0-48c3-b15e-7989de195289`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:54,066::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:32:54,079::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:54,080::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ba3d803-3df0-48c3-b15e-7989de195289`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:32:54,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ba3d803-3df0-48c3-b15e-7989de195289`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:54,080::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:54,080::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:54,080::task::993::Storage.TaskManager.Task::(_decref) Task=`9ba3d803-3df0-48c3-b15e-7989de195289`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:32:54,091::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:32:54,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`112ed6bc-dd31-414e-bdca-75dc999b2f5b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:32:54,092::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:32:54,093::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4282110-3431-4f21-aeae-c3f89cf9c78f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:32:54,093::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:32:54,094::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:32:54,094::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f4282110-3431-4f21-aeae-c3f89cf9c78f`::Granted request >Thread-34::DEBUG::2014-08-11 15:32:54,095::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`112ed6bc-dd31-414e-bdca-75dc999b2f5b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:32:54,095::task::993::Storage.TaskManager.Task::(_decref) Task=`112ed6bc-dd31-414e-bdca-75dc999b2f5b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:32:54,115::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:54,115::task::1191::Storage.TaskManager.Task::(prepare) Task=`112ed6bc-dd31-414e-bdca-75dc999b2f5b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:32:54,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`112ed6bc-dd31-414e-bdca-75dc999b2f5b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:32:54,116::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:32:54,116::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:32:54,116::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:32:54,116::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:32:54,117::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:32:54,117::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:32:54,117::task::993::Storage.TaskManager.Task::(_decref) Task=`112ed6bc-dd31-414e-bdca-75dc999b2f5b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:32:54,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:54,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131396 s, 77.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:32:55,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbb727f7-7069-4159-9054-23ce730f28b9`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:32:55,527::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:32:55,527::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418067', 'lastCheck': '6.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:55,528::task::1191::Storage.TaskManager.Task::(prepare) Task=`dbb727f7-7069-4159-9054-23ce730f28b9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418067', 'lastCheck': '6.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:32:55,528::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbb727f7-7069-4159-9054-23ce730f28b9`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:32:55,528::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:32:55,528::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:32:55,528::task::993::Storage.TaskManager.Task::(_decref) Task=`dbb727f7-7069-4159-9054-23ce730f28b9`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:32:55,550::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:32:56,356::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:56,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133333 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:32:58,409::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:32:58,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133389 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:59,019::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:59,019::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:59,049::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:59,051::__init__::232::IOProcess::(_processLogs) (7243) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:32:59,051::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:32:59,052::__init__::232::IOProcess::(_processLogs) (7243) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:59,072::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:32:59,072::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000467431 s, 973 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:32:59,075::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:59,076::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:59,079::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:59,080::__init__::232::IOProcess::(_processLogs) (7244) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:32:59,081::__init__::232::IOProcess::(_processLogs) (7244) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:59,081::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:59,082::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:59,082::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:59,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:59,085::__init__::232::IOProcess::(_processLogs) (7245) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:59,086::__init__::232::IOProcess::(_processLogs) (7245) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:32:59,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:32:59,089::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:59,091::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:59,091::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:59,091::__init__::232::IOProcess::(_processLogs) (7246) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:59,091::__init__::232::IOProcess::(_processLogs) (7246) Queuing response >Thread-14::DEBUG::2014-08-11 15:32:59,092::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:32:59,093::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:32:59,093::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:32:59,094::__init__::232::IOProcess::(_processLogs) (7247) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:32:59,094::__init__::232::IOProcess::(_processLogs) (7247) Queuing response >Dummy-33::DEBUG::2014-08-11 15:33:00,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:00,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013342 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:02,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:02,572::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0285657 s, 35.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:04,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:04,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:33:04,131::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-115::DEBUG::2014-08-11 15:33:04,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b292112-8699-4860-b192-444bee38d899`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:04,153::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:33:04,153::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:33:04,167::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:04,168::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b292112-8699-4860-b192-444bee38d899`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:04,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b292112-8699-4860-b192-444bee38d899`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:04,168::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:04,168::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:04,168::task::993::Storage.TaskManager.Task::(_decref) Task=`1b292112-8699-4860-b192-444bee38d899`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:33:04,178::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-34::DEBUG::2014-08-11 15:33:04,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd9293b6-8155-41e0-892c-cf5fd04c178d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:33:04,179::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:33:04,180::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eef9a9a3-9a1a-4163-9b99-7e9485be7065`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:33:04,180::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:33:04,181::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:33:04,181::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eef9a9a3-9a1a-4163-9b99-7e9485be7065`::Granted request >Thread-34::DEBUG::2014-08-11 15:33:04,181::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bd9293b6-8155-41e0-892c-cf5fd04c178d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:33:04,182::task::993::Storage.TaskManager.Task::(_decref) Task=`bd9293b6-8155-41e0-892c-cf5fd04c178d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:33:04,201::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:04,202::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd9293b6-8155-41e0-892c-cf5fd04c178d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:04,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd9293b6-8155-41e0-892c-cf5fd04c178d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:33:04,202::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:33:04,202::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:33:04,203::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:33:04,203::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:33:04,203::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:33:04,204::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:33:04,204::task::993::Storage.TaskManager.Task::(_decref) Task=`bd9293b6-8155-41e0-892c-cf5fd04c178d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:33:04,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:04,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132314 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:06,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:06,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132394 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:08,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:08,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132164 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:09,095::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:09,095::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:09,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:09,148::__init__::232::IOProcess::(_processLogs) (7248) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:09,149::__init__::232::IOProcess::(_processLogs) (7248) Queuing response >Thread-16::DEBUG::2014-08-11 15:33:09,149::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:09,168::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:09,169::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:09,171::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:33:09,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428855 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:09,174::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:09,174::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:09,175::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:09,175::__init__::232::IOProcess::(_processLogs) (7249) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:09,176::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7249) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:09,177::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:09,178::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:09,178::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:09,179::__init__::232::IOProcess::(_processLogs) (7250) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:09,179::__init__::232::IOProcess::(_processLogs) (7250) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:09,179::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:09,180::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:09,180::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:09,180::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7251) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:09,181::__init__::232::IOProcess::(_processLogs) (7251) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:09,181::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:09,182::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:09,183::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:09,184::__init__::232::IOProcess::(_processLogs) (7252) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:09,184::__init__::232::IOProcess::(_processLogs) (7252) Queuing response >Thread-115::DEBUG::2014-08-11 15:33:10,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4a15a84-9065-4093-9c3e-3795dc669c90`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:33:10,640::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:33:10,641::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428855', 'lastCheck': '1.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:10,641::task::1191::Storage.TaskManager.Task::(prepare) Task=`b4a15a84-9065-4093-9c3e-3795dc669c90`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000428855', 'lastCheck': '1.5', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:10,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`b4a15a84-9065-4093-9c3e-3795dc669c90`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:10,641::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:10,642::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:10,642::task::993::Storage.TaskManager.Task::(_decref) Task=`b4a15a84-9065-4093-9c3e-3795dc669c90`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:33:10,662::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:33:11,001::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:11,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132999 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:13,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:13,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130904 s, 78.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:14,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:14,179::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:33:14,216::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-115::DEBUG::2014-08-11 15:33:14,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`4de03c1b-6481-4ce1-b802-514c9bc8523b`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:14,253::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:33:14,253::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:33:14,267::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:14,268::task::1191::Storage.TaskManager.Task::(prepare) Task=`4de03c1b-6481-4ce1-b802-514c9bc8523b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:14,268::task::595::Storage.TaskManager.Task::(_updateState) Task=`4de03c1b-6481-4ce1-b802-514c9bc8523b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:14,268::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:14,268::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:14,268::task::993::Storage.TaskManager.Task::(_decref) Task=`4de03c1b-6481-4ce1-b802-514c9bc8523b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:33:14,279::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 15:33:14,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`8354186e-7bc3-42ca-b60e-b49687fff777`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:33:14,280::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:33:14,281::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e20629c-98f7-4bac-995e-c51c3a840d03`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:33:14,281::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:33:14,281::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:33:14,282::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9e20629c-98f7-4bac-995e-c51c3a840d03`::Granted request >Thread-34::DEBUG::2014-08-11 15:33:14,282::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8354186e-7bc3-42ca-b60e-b49687fff777`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:33:14,282::task::993::Storage.TaskManager.Task::(_decref) Task=`8354186e-7bc3-42ca-b60e-b49687fff777`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:33:14,302::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:14,303::task::1191::Storage.TaskManager.Task::(prepare) Task=`8354186e-7bc3-42ca-b60e-b49687fff777`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:14,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`8354186e-7bc3-42ca-b60e-b49687fff777`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:33:14,303::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:33:14,303::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:33:14,304::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:33:14,304::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:33:14,304::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:33:14,304::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:33:14,305::task::993::Storage.TaskManager.Task::(_decref) Task=`8354186e-7bc3-42ca-b60e-b49687fff777`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:33:15,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:15,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131823 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:17,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:17,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132722 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:19,185::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:19,186::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:33:19,187::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:19,221::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:33:19,249::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:19,288::__init__::232::IOProcess::(_processLogs) (7253) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:19,291::__init__::232::IOProcess::(_processLogs) (7253) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:19,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:33:19,294::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000883447 s, 515 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:19,297::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:19,300::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:19,304::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:19,305::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:33:19,307::__init__::232::IOProcess::(_processLogs) (7254) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:19,310::__init__::232::IOProcess::(_processLogs) (7254) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:19,311::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:33:19,313::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:19,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:19,317::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:19,318::__init__::232::IOProcess::(_processLogs) (7255) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:19,320::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7255) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:19,322::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:19,324::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 15:33:19,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0516727 s, 19.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:19,326::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:19,327::__init__::232::IOProcess::(_processLogs) (7256) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:19,328::__init__::232::IOProcess::(_processLogs) (7256) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:19,331::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:19,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:19,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:19,334::__init__::232::IOProcess::(_processLogs) (7257) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:19,335::__init__::232::IOProcess::(_processLogs) (7257) Queuing response >Dummy-33::DEBUG::2014-08-11 15:33:21,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:21,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134509 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:23,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:23,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131429 s, 77.9 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:33:24,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`af0b6005-0a46-472a-9894-7d171f1e06b6`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:33:24,083::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:33:24,084::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:24,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:24,085::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:24,085::__init__::232::IOProcess::(_processLogs) (7258) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:33:24,086::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7258) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:24,087::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:24,087::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:33:24,087::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:33:24,088::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:33:24,089::task::1191::Storage.TaskManager.Task::(prepare) Task=`af0b6005-0a46-472a-9894-7d171f1e06b6`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:33:24,090::__init__::232::IOProcess::(_processLogs) (7259) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:33:24,090::task::595::Storage.TaskManager.Task::(_updateState) Task=`af0b6005-0a46-472a-9894-7d171f1e06b6`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:33:24,091::__init__::232::IOProcess::(_processLogs) (7259) Queuing response >Thread-122::DEBUG::2014-08-11 15:33:24,092::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:33:24,093::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:33:24,093::task::993::Storage.TaskManager.Task::(_decref) Task=`af0b6005-0a46-472a-9894-7d171f1e06b6`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:33:24,317::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-115::DEBUG::2014-08-11 15:33:24,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d2e697c-6279-4437-bd11-96a6961da134`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:24,321::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:33:24,349::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:24,352::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:24,353::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:33:24,365::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:24,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d2e697c-6279-4437-bd11-96a6961da134`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:24,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d2e697c-6279-4437-bd11-96a6961da134`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:24,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:24,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:24,367::task::993::Storage.TaskManager.Task::(_decref) Task=`3d2e697c-6279-4437-bd11-96a6961da134`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:33:24,377::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-34::DEBUG::2014-08-11 15:33:24,378::task::595::Storage.TaskManager.Task::(_updateState) Task=`4070aa2e-88f4-493c-9cec-1870008366cf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:33:24,378::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:33:24,379::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4143a8f8-b0ca-484c-bc6a-dd5dfd3ee835`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:33:24,379::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:33:24,380::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:33:24,380::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4143a8f8-b0ca-484c-bc6a-dd5dfd3ee835`::Granted request >Thread-34::DEBUG::2014-08-11 15:33:24,380::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4070aa2e-88f4-493c-9cec-1870008366cf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:33:24,381::task::993::Storage.TaskManager.Task::(_decref) Task=`4070aa2e-88f4-493c-9cec-1870008366cf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:33:24,401::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:24,401::task::1191::Storage.TaskManager.Task::(prepare) Task=`4070aa2e-88f4-493c-9cec-1870008366cf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:24,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`4070aa2e-88f4-493c-9cec-1870008366cf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:33:24,402::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:33:24,403::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:33:24,404::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:33:24,404::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:33:24,405::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:33:24,405::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:33:24,406::task::993::Storage.TaskManager.Task::(_decref) Task=`4070aa2e-88f4-493c-9cec-1870008366cf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:33:25,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:25,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133551 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:33:25,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`c661ce00-eade-4c6c-ad6d-09aa0d6b497e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:33:25,747::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:33:25,748::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000883447', 'lastCheck': '6.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:25,748::task::1191::Storage.TaskManager.Task::(prepare) Task=`c661ce00-eade-4c6c-ad6d-09aa0d6b497e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000883447', 'lastCheck': '6.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:25,748::task::595::Storage.TaskManager.Task::(_updateState) Task=`c661ce00-eade-4c6c-ad6d-09aa0d6b497e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:25,749::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:25,749::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:25,749::task::993::Storage.TaskManager.Task::(_decref) Task=`c661ce00-eade-4c6c-ad6d-09aa0d6b497e`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:33:25,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:33:27,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:27,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133367 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:29,339::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:29,340::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:29,349::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:33:29,350::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:33:29,350::__init__::232::IOProcess::(_processLogs) (7260) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:29,352::__init__::232::IOProcess::(_processLogs) (7260) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:29,371::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:33:29,374::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00153355 s, 297 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:29,375::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:33:29,377::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:29,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:29,380::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:29,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:33:29,382::__init__::232::IOProcess::(_processLogs) (7261) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:29,384::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7261) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:29,386::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:29,386::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:29,386::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:29,387::__init__::232::IOProcess::(_processLogs) (7262) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:29,387::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7262) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:29,388::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:29,389::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:29,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:29,389::__init__::232::IOProcess::(_processLogs) (7263) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:29,390::__init__::232::IOProcess::(_processLogs) (7263) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:29,390::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:29,390::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:29,391::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:29,391::__init__::232::IOProcess::(_processLogs) (7264) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:29,391::__init__::232::IOProcess::(_processLogs) (7264) Queuing response >Dummy-33::DEBUG::2014-08-11 15:33:29,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:29,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134213 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:31,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:31,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134078 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:33,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:33,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134728 s, 76.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:34,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:34,391::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:33:34,419::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:33:34,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`01486a8e-74a8-4ae9-9b21-c642676d05ed`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:33:34,452::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:34,454::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:33:34,467::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:34,469::task::1191::Storage.TaskManager.Task::(prepare) Task=`01486a8e-74a8-4ae9-9b21-c642676d05ed`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:34,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`01486a8e-74a8-4ae9-9b21-c642676d05ed`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:34,471::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:34,472::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:34,474::task::993::Storage.TaskManager.Task::(_decref) Task=`01486a8e-74a8-4ae9-9b21-c642676d05ed`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:33:34,486::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:33:34,487::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6e70425-cddc-44ac-a7e7-e7a8e26f4cab`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:33:34,487::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:33:34,488::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6422964b-c5bf-4b1c-9abb-fa59eadb61ee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:33:34,488::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:33:34,489::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:33:34,489::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6422964b-c5bf-4b1c-9abb-fa59eadb61ee`::Granted request >Thread-34::DEBUG::2014-08-11 15:33:34,489::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e6e70425-cddc-44ac-a7e7-e7a8e26f4cab`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:33:34,490::task::993::Storage.TaskManager.Task::(_decref) Task=`e6e70425-cddc-44ac-a7e7-e7a8e26f4cab`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:33:34,509::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:34,510::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6e70425-cddc-44ac-a7e7-e7a8e26f4cab`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852959162368', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:34,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6e70425-cddc-44ac-a7e7-e7a8e26f4cab`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:33:34,510::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:33:34,510::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:33:34,511::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:33:34,511::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:33:34,511::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:33:34,512::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:33:34,512::task::993::Storage.TaskManager.Task::(_decref) Task=`e6e70425-cddc-44ac-a7e7-e7a8e26f4cab`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:33:35,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:35,781::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131358 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:37,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:37,836::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134524 s, 76.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:39,393::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:39,394::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:33:39,395::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:39,468::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:39,470::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:39,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:33:39,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:39,473::__init__::232::IOProcess::(_processLogs) (7265) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:33:39,474::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414328 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:39,474::__init__::232::IOProcess::(_processLogs) (7265) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:39,476::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:39,477::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:39,477::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:39,477::__init__::232::IOProcess::(_processLogs) (7266) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:39,478::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7266) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:39,479::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:39,480::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:39,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:39,481::__init__::232::IOProcess::(_processLogs) (7267) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:39,481::__init__::232::IOProcess::(_processLogs) (7267) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:39,481::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:39,481::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:39,482::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:39,482::__init__::232::IOProcess::(_processLogs) (7268) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:39,483::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7268) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:39,483::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:39,484::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:39,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:39,485::__init__::232::IOProcess::(_processLogs) (7269) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:39,486::__init__::232::IOProcess::(_processLogs) (7269) Queuing response >Dummy-33::DEBUG::2014-08-11 15:33:39,864::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:39,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132374 s, 77.4 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:33:40,863::task::595::Storage.TaskManager.Task::(_updateState) Task=`c52ddacb-c78b-41a2-b44c-b4c06cbdaf9d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:33:40,864::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:33:40,864::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414328', 'lastCheck': '1.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:40,864::task::1191::Storage.TaskManager.Task::(prepare) Task=`c52ddacb-c78b-41a2-b44c-b4c06cbdaf9d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414328', 'lastCheck': '1.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:40,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`c52ddacb-c78b-41a2-b44c-b4c06cbdaf9d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:40,865::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:40,865::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:40,865::task::993::Storage.TaskManager.Task::(_decref) Task=`c52ddacb-c78b-41a2-b44c-b4c06cbdaf9d`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:33:40,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:33:41,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:41,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135125 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:43,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:44,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132608 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:44,479::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:44,480::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:33:44,525::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:33:44,551::task::595::Storage.TaskManager.Task::(_updateState) Task=`777a4ba1-ec18-49b7-b477-38c404c5e191`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:33:44,553::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:44,553::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:33:44,567::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:44,567::task::1191::Storage.TaskManager.Task::(prepare) Task=`777a4ba1-ec18-49b7-b477-38c404c5e191`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:44,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`777a4ba1-ec18-49b7-b477-38c404c5e191`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:44,568::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:44,568::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:44,568::task::993::Storage.TaskManager.Task::(_decref) Task=`777a4ba1-ec18-49b7-b477-38c404c5e191`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:33:44,579::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:33:44,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`af9fa0aa-e50e-4876-a2e6-9207af92e7ff`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:33:44,580::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:33:44,581::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`04e5aab2-df27-437e-82ec-70a84a0ab2bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:33:44,581::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:33:44,581::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:33:44,582::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`04e5aab2-df27-437e-82ec-70a84a0ab2bc`::Granted request >Thread-34::DEBUG::2014-08-11 15:33:44,582::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`af9fa0aa-e50e-4876-a2e6-9207af92e7ff`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:33:44,582::task::993::Storage.TaskManager.Task::(_decref) Task=`af9fa0aa-e50e-4876-a2e6-9207af92e7ff`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:33:44,602::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852956999680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:44,602::task::1191::Storage.TaskManager.Task::(prepare) Task=`af9fa0aa-e50e-4876-a2e6-9207af92e7ff`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852956999680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:44,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`af9fa0aa-e50e-4876-a2e6-9207af92e7ff`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:33:44,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:33:44,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:33:44,603::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:33:44,604::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:33:44,604::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:33:44,604::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:33:44,604::task::993::Storage.TaskManager.Task::(_decref) Task=`af9fa0aa-e50e-4876-a2e6-9207af92e7ff`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:33:46,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:46,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135838 s, 75.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:48,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:48,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132443 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:49,486::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:49,487::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:33:49,488::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:33:49,549::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:49,568::__init__::232::IOProcess::(_processLogs) (7270) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:49,571::__init__::232::IOProcess::(_processLogs) (7270) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:49,571::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:33:49,573::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00105687 s, 431 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:49,576::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:49,577::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:49,579::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:49,579::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:49,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:33:49,583::__init__::232::IOProcess::(_processLogs) (7271) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:49,583::__init__::232::IOProcess::(_processLogs) (7271) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:49,583::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:49,584::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:49,585::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:49,585::__init__::232::IOProcess::(_processLogs) (7272) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:49,585::__init__::232::IOProcess::(_processLogs) (7272) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:49,586::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:49,586::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:49,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:49,587::__init__::232::IOProcess::(_processLogs) (7273) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:49,587::__init__::232::IOProcess::(_processLogs) (7273) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:49,588::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:49,588::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:49,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:49,590::__init__::232::IOProcess::(_processLogs) (7274) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:49,590::__init__::232::IOProcess::(_processLogs) (7274) Queuing response >Dummy-33::DEBUG::2014-08-11 15:33:50,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:50,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135299 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:52,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:52,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131982 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:54,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:54,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028219 s, 36.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:54,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:54,586::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:33:54,617::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:33:54,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`308062c0-ecd2-4781-a710-bd19bc73e22f`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:33:54,653::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:54,653::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:33:54,667::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:54,667::task::1191::Storage.TaskManager.Task::(prepare) Task=`308062c0-ecd2-4781-a710-bd19bc73e22f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:33:54,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`308062c0-ecd2-4781-a710-bd19bc73e22f`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:54,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:54,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:54,668::task::993::Storage.TaskManager.Task::(_decref) Task=`308062c0-ecd2-4781-a710-bd19bc73e22f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:33:54,679::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:33:54,680::task::595::Storage.TaskManager.Task::(_updateState) Task=`c907061f-a186-4b65-b327-6b7ca6d6ec68`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:33:54,680::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:33:54,681::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dd3efc09-7065-4254-a38f-15d06f1a2377`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:33:54,681::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:33:54,681::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:33:54,682::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dd3efc09-7065-4254-a38f-15d06f1a2377`::Granted request >Thread-34::DEBUG::2014-08-11 15:33:54,682::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c907061f-a186-4b65-b327-6b7ca6d6ec68`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:33:54,682::task::993::Storage.TaskManager.Task::(_decref) Task=`c907061f-a186-4b65-b327-6b7ca6d6ec68`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:33:54,702::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852956999680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:54,702::task::1191::Storage.TaskManager.Task::(prepare) Task=`c907061f-a186-4b65-b327-6b7ca6d6ec68`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852956999680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:33:54,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`c907061f-a186-4b65-b327-6b7ca6d6ec68`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:33:54,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:33:54,703::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:33:54,704::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:33:54,704::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:33:54,704::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:33:54,704::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:33:54,705::task::993::Storage.TaskManager.Task::(_decref) Task=`c907061f-a186-4b65-b327-6b7ca6d6ec68`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:33:55,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0e4f156-b355-45f8-a05a-f7c80f287510`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:33:55,983::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:33:55,983::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00105687', 'lastCheck': '6.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:55,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`c0e4f156-b355-45f8-a05a-f7c80f287510`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00105687', 'lastCheck': '6.4', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:33:55,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0e4f156-b355-45f8-a05a-f7c80f287510`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:33:55,984::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:33:55,984::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:33:55,985::task::993::Storage.TaskManager.Task::(_decref) Task=`c0e4f156-b355-45f8-a05a-f7c80f287510`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:33:56,005::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:33:56,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:56,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137443 s, 74.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:33:58,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:33:58,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132927 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:59,591::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:59,591::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:59,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:59,649::__init__::232::IOProcess::(_processLogs) (7275) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:59,650::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7275) Queuing response >Thread-16::DEBUG::2014-08-11 15:33:59,651::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:59,670::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:59,672::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:33:59,673::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:33:59,675::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421693 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:33:59,676::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:59,676::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:59,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:59,677::__init__::232::IOProcess::(_processLogs) (7276) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:33:59,678::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7276) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:59,679::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:59,680::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:59,680::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:59,681::__init__::232::IOProcess::(_processLogs) (7277) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:59,681::__init__::232::IOProcess::(_processLogs) (7277) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:59,681::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:59,682::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:59,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:59,683::__init__::232::IOProcess::(_processLogs) (7278) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:59,683::__init__::232::IOProcess::(_processLogs) (7278) Queuing response >Thread-14::DEBUG::2014-08-11 15:33:59,683::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:33:59,684::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:33:59,684::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:33:59,685::__init__::232::IOProcess::(_processLogs) (7279) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:33:59,686::__init__::232::IOProcess::(_processLogs) (7279) Queuing response >Dummy-33::DEBUG::2014-08-11 15:34:00,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:00,468::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132823 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:02,491::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:02,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0272403 s, 37.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:04,680::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:04,681::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:34:04,722::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:34:04,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c100905-0842-45ae-adee-28c1732c768a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:04,754::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:04,754::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:34:04,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-115::INFO::2014-08-11 15:34:04,779::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:04,780::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c100905-0842-45ae-adee-28c1732c768a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:04,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c100905-0842-45ae-adee-28c1732c768a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:04,780::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:04,780::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:04,781::task::993::Storage.TaskManager.Task::(_decref) Task=`9c100905-0842-45ae-adee-28c1732c768a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:34:04,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133183 s, 76.9 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 15:34:04,799::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:34:04,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e0f816a-c376-4287-90ec-347e1dcb2733`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:34:04,803::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:34:04,806::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8a9bc47d-bf13-46fc-9a57-11c55de06995`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:34:04,807::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:34:04,808::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:34:04,809::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8a9bc47d-bf13-46fc-9a57-11c55de06995`::Granted request >Thread-34::DEBUG::2014-08-11 15:34:04,812::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4e0f816a-c376-4287-90ec-347e1dcb2733`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:34:04,813::task::993::Storage.TaskManager.Task::(_decref) Task=`4e0f816a-c376-4287-90ec-347e1dcb2733`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:34:04,856::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852956999680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:04,857::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e0f816a-c376-4287-90ec-347e1dcb2733`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852956999680', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:04,858::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e0f816a-c376-4287-90ec-347e1dcb2733`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:34:04,859::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:34:04,860::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:34:04,861::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:34:04,863::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:34:04,864::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:34:04,865::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:34:04,866::task::993::Storage.TaskManager.Task::(_decref) Task=`4e0f816a-c376-4287-90ec-347e1dcb2733`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:34:06,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:06,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132992 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:08,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:08,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132944 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:34:09,686::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:09,687::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:34:09,688::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:34:09,750::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:09,769::__init__::232::IOProcess::(_processLogs) (7280) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:09,770::__init__::232::IOProcess::(_processLogs) (7280) Queuing response >Thread-16::DEBUG::2014-08-11 15:34:09,771::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000408808 s, 1.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:09,769::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:09,772::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:09,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:09,775::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:09,776::__init__::232::IOProcess::(_processLogs) (7281) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:09,777::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:09,778::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7281) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:09,780::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:09,781::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:09,783::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:09,784::__init__::232::IOProcess::(_processLogs) (7282) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:09,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:09,785::__init__::232::IOProcess::(_processLogs) (7282) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:09,787::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:09,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:09,789::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:09,789::__init__::232::IOProcess::(_processLogs) (7283) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:09,790::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7283) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:09,791::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:09,791::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:09,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:09,793::__init__::232::IOProcess::(_processLogs) (7284) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:09,793::__init__::232::IOProcess::(_processLogs) (7284) Queuing response >Dummy-33::DEBUG::2014-08-11 15:34:10,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:11,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135504 s, 75.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:34:11,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f7eda34-5e0e-496b-a93e-2cf3463770d8`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:11,111::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:34:11,112::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000408808', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:11,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`2f7eda34-5e0e-496b-a93e-2cf3463770d8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000408808', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:11,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`2f7eda34-5e0e-496b-a93e-2cf3463770d8`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:11,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:11,113::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:11,113::task::993::Storage.TaskManager.Task::(_decref) Task=`2f7eda34-5e0e-496b-a93e-2cf3463770d8`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:34:11,137::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:34:13,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:13,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132728 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:14,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:14,850::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:14,851::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:34:14,882::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:34:14,883::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed7539db-a920-4190-94f8-82657a88e407`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:14,883::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:34:14,895::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:14,896::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed7539db-a920-4190-94f8-82657a88e407`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:14,896::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed7539db-a920-4190-94f8-82657a88e407`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:14,896::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:14,897::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:14,897::task::993::Storage.TaskManager.Task::(_decref) Task=`ed7539db-a920-4190-94f8-82657a88e407`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:34:14,908::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:34:14,909::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb1c1ae2-a89d-4284-870b-35d197420574`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:34:14,909::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:34:14,910::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`749db375-8f58-4fde-bf16-b29ad0f82cb9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:34:14,910::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:34:14,911::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:34:14,911::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`749db375-8f58-4fde-bf16-b29ad0f82cb9`::Granted request >Thread-34::DEBUG::2014-08-11 15:34:14,912::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fb1c1ae2-a89d-4284-870b-35d197420574`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:34:14,912::task::993::Storage.TaskManager.Task::(_decref) Task=`fb1c1ae2-a89d-4284-870b-35d197420574`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:34:14,932::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852955820032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:14,932::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb1c1ae2-a89d-4284-870b-35d197420574`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852955820032', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:14,932::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb1c1ae2-a89d-4284-870b-35d197420574`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:34:14,932::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:34:14,933::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:34:14,933::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:34:14,933::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:34:14,934::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:34:14,934::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:34:14,934::task::993::Storage.TaskManager.Task::(_decref) Task=`fb1c1ae2-a89d-4284-870b-35d197420574`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:34:15,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:15,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131919 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:17,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:17,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131279 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:19,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:19,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0210751 s, 48.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:34:19,794::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:19,795::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:34:19,796::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:34:19,796::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:19,868::__init__::232::IOProcess::(_processLogs) (7285) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:19,870::__init__::232::IOProcess::(_processLogs) (7285) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:19,871::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:34:19,873::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00108012 s, 421 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:34:19,876::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:19,877::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:19,879::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:19,879::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:19,881::__init__::232::IOProcess::(_processLogs) (7286) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:19,882::__init__::232::IOProcess::(_processLogs) (7286) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:19,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:19,885::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:19,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:19,888::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:19,889::__init__::232::IOProcess::(_processLogs) (7287) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:19,889::__init__::232::IOProcess::(_processLogs) (7287) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:19,890::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:19,890::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:19,890::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:19,891::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7288) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:19,894::__init__::232::IOProcess::(_processLogs) (7288) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:19,895::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:19,895::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:19,896::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:19,897::__init__::232::IOProcess::(_processLogs) (7289) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:19,897::__init__::232::IOProcess::(_processLogs) (7289) Queuing response >Dummy-33::DEBUG::2014-08-11 15:34:21,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:21,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132232 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:23,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:23,366::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133978 s, 76.4 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:34:24,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b42f769-f537-4f7c-9cb1-7e2346a09449`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:34:24,199::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:34:24,200::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:24,201::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:24,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:24,201::__init__::232::IOProcess::(_processLogs) (7290) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:34:24,202::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7290) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:24,203::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:24,204::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:34:24,204::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:34:24,205::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:34:24,205::task::1191::Storage.TaskManager.Task::(prepare) Task=`3b42f769-f537-4f7c-9cb1-7e2346a09449`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:34:24,206::__init__::232::IOProcess::(_processLogs) (7291) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:34:24,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`3b42f769-f537-4f7c-9cb1-7e2346a09449`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:34:24,208::__init__::232::IOProcess::(_processLogs) (7291) Queuing response >Thread-122::DEBUG::2014-08-11 15:34:24,208::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:34:24,209::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:34:24,209::task::993::Storage.TaskManager.Task::(_decref) Task=`3b42f769-f537-4f7c-9cb1-7e2346a09449`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:24,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:24,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:34:24,945::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:34:24,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`30099235-17bd-414f-a453-bfc4b4d8b69e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:24,954::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:24,956::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:34:24,968::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:24,969::task::1191::Storage.TaskManager.Task::(prepare) Task=`30099235-17bd-414f-a453-bfc4b4d8b69e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:24,969::task::595::Storage.TaskManager.Task::(_updateState) Task=`30099235-17bd-414f-a453-bfc4b4d8b69e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:24,969::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:24,969::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:24,970::task::993::Storage.TaskManager.Task::(_decref) Task=`30099235-17bd-414f-a453-bfc4b4d8b69e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:34:24,981::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:34:24,981::task::595::Storage.TaskManager.Task::(_updateState) Task=`4dbc69b4-a1c9-404a-8a2f-dab0f3c0548a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:34:24,982::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:34:24,982::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`795c5323-3f24-4f65-b21f-88e78a8b4a6f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:34:24,983::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:34:24,983::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:34:24,983::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`795c5323-3f24-4f65-b21f-88e78a8b4a6f`::Granted request >Thread-34::DEBUG::2014-08-11 15:34:24,984::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4dbc69b4-a1c9-404a-8a2f-dab0f3c0548a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:34:24,984::task::993::Storage.TaskManager.Task::(_decref) Task=`4dbc69b4-a1c9-404a-8a2f-dab0f3c0548a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:34:25,004::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852954836992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:25,004::task::1191::Storage.TaskManager.Task::(prepare) Task=`4dbc69b4-a1c9-404a-8a2f-dab0f3c0548a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852954836992', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:25,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`4dbc69b4-a1c9-404a-8a2f-dab0f3c0548a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:34:25,005::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:34:25,005::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:34:25,005::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:34:25,006::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:34:25,006::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:34:25,006::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:34:25,006::task::993::Storage.TaskManager.Task::(_decref) Task=`4dbc69b4-a1c9-404a-8a2f-dab0f3c0548a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:34:25,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:25,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133079 s, 76.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:34:26,233::task::595::Storage.TaskManager.Task::(_updateState) Task=`60c8da61-9fce-46fd-927a-07e900607202`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:26,234::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:34:26,234::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00108012', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:26,234::task::1191::Storage.TaskManager.Task::(prepare) Task=`60c8da61-9fce-46fd-927a-07e900607202`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00108012', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:26,235::task::595::Storage.TaskManager.Task::(_updateState) Task=`60c8da61-9fce-46fd-927a-07e900607202`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:26,235::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:26,235::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:26,235::task::993::Storage.TaskManager.Task::(_decref) Task=`60c8da61-9fce-46fd-927a-07e900607202`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:34:26,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:34:27,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:27,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133207 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:29,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:29,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132288 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:34:29,898::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:29,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:29,899::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:29,900::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7292) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:29,900::__init__::232::IOProcess::(_processLogs) (7292) Queuing response >Thread-16::DEBUG::2014-08-11 15:34:29,901::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:29,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:29,970::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:29,971::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:34:29,974::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000429759 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:34:29,975::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:29,975::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:29,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:29,976::__init__::232::IOProcess::(_processLogs) (7293) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:29,976::__init__::232::IOProcess::(_processLogs) DEBUG|(7293) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:29,978::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:29,978::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:29,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:29,979::__init__::232::IOProcess::(_processLogs) (7294) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:29,980::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7294) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:29,980::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:29,981::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:29,982::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:29,982::__init__::232::IOProcess::(_processLogs) (7295) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:29,982::__init__::232::IOProcess::(_processLogs) (7295) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:29,983::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:29,983::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:29,983::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:29,983::__init__::232::IOProcess::(_processLogs) (7296) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:29,984::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7296) Queuing response >Dummy-33::DEBUG::2014-08-11 15:34:31,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:31,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0140856 s, 72.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:33,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:33,693::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013248 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:34,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:34,979::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:34:35,019::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:34:35,053::task::595::Storage.TaskManager.Task::(_updateState) Task=`92b60126-3dc8-4885-ba4d-ebdc533a3165`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:35,054::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:35,056::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:34:35,069::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:35,070::task::1191::Storage.TaskManager.Task::(prepare) Task=`92b60126-3dc8-4885-ba4d-ebdc533a3165`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:35,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`92b60126-3dc8-4885-ba4d-ebdc533a3165`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:35,070::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:35,070::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:35,071::task::993::Storage.TaskManager.Task::(_decref) Task=`92b60126-3dc8-4885-ba4d-ebdc533a3165`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:34:35,081::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:34:35,082::task::595::Storage.TaskManager.Task::(_updateState) Task=`a32686de-2c0a-46f3-a122-29764bcd6c49`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:34:35,082::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:34:35,083::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5fae482c-e226-464e-8e6d-474cee327a07`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:34:35,083::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:34:35,084::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:34:35,084::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5fae482c-e226-464e-8e6d-474cee327a07`::Granted request >Thread-34::DEBUG::2014-08-11 15:34:35,084::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a32686de-2c0a-46f3-a122-29764bcd6c49`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:34:35,085::task::993::Storage.TaskManager.Task::(_decref) Task=`a32686de-2c0a-46f3-a122-29764bcd6c49`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:34:35,104::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952674304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:35,105::task::1191::Storage.TaskManager.Task::(prepare) Task=`a32686de-2c0a-46f3-a122-29764bcd6c49`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952674304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:35,105::task::595::Storage.TaskManager.Task::(_updateState) Task=`a32686de-2c0a-46f3-a122-29764bcd6c49`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:34:35,105::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:34:35,105::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:34:35,106::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:34:35,106::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:34:35,106::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:34:35,107::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:34:35,107::task::993::Storage.TaskManager.Task::(_decref) Task=`a32686de-2c0a-46f3-a122-29764bcd6c49`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:34:35,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:35,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132428 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:37,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:37,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132092 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:39,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:39,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132597 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:34:39,988::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:40,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:40,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:40,051::__init__::232::IOProcess::(_processLogs) (7297) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:40,052::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7297) Queuing response >Thread-16::DEBUG::2014-08-11 15:34:40,054::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:34:40,074::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000593759 s, 766 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:40,075::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:40,077::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:40,078::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:40,080::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:40,081::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:40,082::__init__::232::IOProcess::(_processLogs) (7298) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:40,084::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7298) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:40,086::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:40,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:40,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:40,089::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:40,090::__init__::232::IOProcess::(_processLogs) (7299) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:40,091::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7299) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:40,091::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:40,092::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:40,092::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:40,092::__init__::232::IOProcess::(_processLogs) (7300) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:40,093::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7300) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:40,094::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:40,094::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:40,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:40,096::__init__::232::IOProcess::(_processLogs) (7301) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:40,096::__init__::232::IOProcess::(_processLogs) (7301) Queuing response >Thread-115::DEBUG::2014-08-11 15:34:41,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6f064a8-78a4-4fc5-9eb7-2091dbb6a32b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:41,358::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:34:41,358::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000593759', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:41,358::task::1191::Storage.TaskManager.Task::(prepare) Task=`a6f064a8-78a4-4fc5-9eb7-2091dbb6a32b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000593759', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:41,358::task::595::Storage.TaskManager.Task::(_updateState) Task=`a6f064a8-78a4-4fc5-9eb7-2091dbb6a32b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:41,359::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:41,359::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:41,359::task::993::Storage.TaskManager.Task::(_decref) Task=`a6f064a8-78a4-4fc5-9eb7-2091dbb6a32b`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:34:41,379::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:34:41,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:41,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132068 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:43,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:43,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134598 s, 76.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:45,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:45,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:45,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:34:45,126::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:34:45,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`d83fc66c-7737-4db7-bae5-0dbfef429b72`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:45,151::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:34:45,164::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:45,164::task::1191::Storage.TaskManager.Task::(prepare) Task=`d83fc66c-7737-4db7-bae5-0dbfef429b72`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:45,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`d83fc66c-7737-4db7-bae5-0dbfef429b72`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:45,165::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:45,165::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:45,165::task::993::Storage.TaskManager.Task::(_decref) Task=`d83fc66c-7737-4db7-bae5-0dbfef429b72`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:34:45,176::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:34:45,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d854fe6-dfee-4c9c-8bd9-f844fcd012a3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:34:45,177::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:34:45,178::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`04ef7a7a-1d0a-4cf9-86e0-17e25dcc0172`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:34:45,178::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:34:45,178::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:34:45,178::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`04ef7a7a-1d0a-4cf9-86e0-17e25dcc0172`::Granted request >Thread-34::DEBUG::2014-08-11 15:34:45,179::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4d854fe6-dfee-4c9c-8bd9-f844fcd012a3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:34:45,179::task::993::Storage.TaskManager.Task::(_decref) Task=`4d854fe6-dfee-4c9c-8bd9-f844fcd012a3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:34:45,199::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952674304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:45,199::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d854fe6-dfee-4c9c-8bd9-f844fcd012a3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952674304', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:45,200::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d854fe6-dfee-4c9c-8bd9-f844fcd012a3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:34:45,200::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:34:45,200::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:34:45,201::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:34:45,201::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:34:45,201::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:34:45,201::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:34:45,202::task::993::Storage.TaskManager.Task::(_decref) Task=`4d854fe6-dfee-4c9c-8bd9-f844fcd012a3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:34:45,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:46,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132073 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:48,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:48,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134571 s, 76.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:34:50,096::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:50,098::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:34:50,099::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:34:50,099::__init__::232::IOProcess::(_processLogs) Extracting request information... >Dummy-33::DEBUG::2014-08-11 15:34:50,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:34:50,101::__init__::232::IOProcess::(_processLogs) (7302) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:34:50,138::__init__::232::IOProcess::(_processLogs) (7302) Queuing response >Thread-16::DEBUG::2014-08-11 15:34:50,141::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000900531 s, 505 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:50,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:50,155::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:50,174::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:50,179::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:50,182::__init__::232::IOProcess::(_processLogs) (7303) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:50,182::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:34:50,184::__init__::232::IOProcess::(_processLogs) (7303) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:50,188::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:50,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:50,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:50,190::__init__::232::IOProcess::(_processLogs) (7304) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:50,191::__init__::232::IOProcess::(_processLogs) (7304) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:50,192::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:50,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:50,193::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:50,193::__init__::232::IOProcess::(_processLogs) (7305) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:50,193::__init__::232::IOProcess::(_processLogs) (7305) Queuing response >Thread-14::DEBUG::2014-08-11 15:34:50,195::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:34:50,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:34:50,196::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:34:50,196::__init__::232::IOProcess::(_processLogs) (7306) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:34:50,197::__init__::232::IOProcess::(_processLogs) (7306) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:50,203::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:34:50,208::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0640457 s, 16.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:52,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:52,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131409 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:34:54,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:54,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131372 s, 77.9 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:34:55,216::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:55,216::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:34:55,218::task::595::Storage.TaskManager.Task::(_updateState) Task=`95cdf7a7-06c6-467d-8362-c52e24972a4c`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:55,253::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:55,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:34:55,257::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:34:55,285::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:55,286::task::1191::Storage.TaskManager.Task::(prepare) Task=`95cdf7a7-06c6-467d-8362-c52e24972a4c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:34:55,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`95cdf7a7-06c6-467d-8362-c52e24972a4c`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:55,287::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:55,287::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:55,287::task::993::Storage.TaskManager.Task::(_decref) Task=`95cdf7a7-06c6-467d-8362-c52e24972a4c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:34:55,306::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:34:55,309::task::595::Storage.TaskManager.Task::(_updateState) Task=`850379a9-a769-4a00-a3f1-4bbb0dccfa01`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:34:55,309::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:34:55,312::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`76a49d75-6df5-4c9a-8a65-aeb806b15d71`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:34:55,313::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:34:55,313::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:34:55,314::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`76a49d75-6df5-4c9a-8a65-aeb806b15d71`::Granted request >Thread-34::DEBUG::2014-08-11 15:34:55,315::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`850379a9-a769-4a00-a3f1-4bbb0dccfa01`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:34:55,315::task::993::Storage.TaskManager.Task::(_decref) Task=`850379a9-a769-4a00-a3f1-4bbb0dccfa01`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:34:55,341::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952608768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:55,341::task::1191::Storage.TaskManager.Task::(prepare) Task=`850379a9-a769-4a00-a3f1-4bbb0dccfa01`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952608768', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:34:55,341::task::595::Storage.TaskManager.Task::(_updateState) Task=`850379a9-a769-4a00-a3f1-4bbb0dccfa01`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:34:55,342::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:34:55,342::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:34:55,342::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:34:55,343::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:34:55,343::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:34:55,343::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:34:55,343::task::993::Storage.TaskManager.Task::(_decref) Task=`850379a9-a769-4a00-a3f1-4bbb0dccfa01`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:34:56,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:56,389::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134715 s, 76.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:34:56,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f435c37-edc2-40cb-b97f-b92edc991bc1`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:34:56,476::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:34:56,476::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000900531', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:56,477::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f435c37-edc2-40cb-b97f-b92edc991bc1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000900531', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:34:56,477::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f435c37-edc2-40cb-b97f-b92edc991bc1`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:34:56,477::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:34:56,477::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:34:56,477::task::993::Storage.TaskManager.Task::(_decref) Task=`4f435c37-edc2-40cb-b97f-b92edc991bc1`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:34:56,499::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:34:58,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:34:58,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132748 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:00,200::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:00,200::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:00,200::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:00,201::__init__::232::IOProcess::(_processLogs) (7307) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:00,201::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7307) Queuing response >Thread-16::DEBUG::2014-08-11 15:35:00,203::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:35:00,221::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00044312 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:00,222::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:00,252::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:00,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:00,254::__init__::232::IOProcess::(_processLogs) (7308) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:00,254::__init__::232::IOProcess::(_processLogs) (7308) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:00,254::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:00,255::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:00,255::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:00,255::__init__::232::IOProcess::(_processLogs) (7309) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:00,256::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7309) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:00,256::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:00,257::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:00,257::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:00,257::__init__::232::IOProcess::(_processLogs) (7310) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:00,258::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7310) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:00,259::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:00,259::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:00,259::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:00,260::__init__::232::IOProcess::(_processLogs) (7311) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:00,260::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7311) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:00,280::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:00,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:00,282::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:00,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:00,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132389 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:02,522::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:02,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0304567 s, 33.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:04,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:04,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131546 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:05,288::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:05,289::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:05,352::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:35:05,355::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:35:05,356::task::595::Storage.TaskManager.Task::(_updateState) Task=`65ee7a51-d75d-4fee-af64-e1cea5ef0bbc`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:05,357::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:35:05,369::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:05,370::task::1191::Storage.TaskManager.Task::(prepare) Task=`65ee7a51-d75d-4fee-af64-e1cea5ef0bbc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:05,370::task::595::Storage.TaskManager.Task::(_updateState) Task=`65ee7a51-d75d-4fee-af64-e1cea5ef0bbc`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:05,370::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:05,370::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:05,371::task::993::Storage.TaskManager.Task::(_decref) Task=`65ee7a51-d75d-4fee-af64-e1cea5ef0bbc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:35:05,382::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:35:05,383::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fdb89b3-e680-4b64-a601-a0bf4f8f864e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:35:05,383::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:35:05,384::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fac71c27-a2ba-4867-b666-597c209488a7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:35:05,384::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:35:05,384::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:35:05,385::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fac71c27-a2ba-4867-b666-597c209488a7`::Granted request >Thread-34::DEBUG::2014-08-11 15:35:05,385::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4fdb89b3-e680-4b64-a601-a0bf4f8f864e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:35:05,386::task::993::Storage.TaskManager.Task::(_decref) Task=`4fdb89b3-e680-4b64-a601-a0bf4f8f864e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:35:05,405::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:05,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`4fdb89b3-e680-4b64-a601-a0bf4f8f864e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:05,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fdb89b3-e680-4b64-a601-a0bf4f8f864e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:35:05,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:35:05,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:35:05,407::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:35:05,407::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:35:05,407::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:35:05,408::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:35:05,408::task::993::Storage.TaskManager.Task::(_decref) Task=`4fdb89b3-e680-4b64-a601-a0bf4f8f864e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:35:06,811::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:06,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132371 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:08,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:08,898::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132715 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:10,262::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:10,262::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:10,262::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:10,263::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7312) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:35:10,265::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:35:10,265::__init__::232::IOProcess::(_processLogs) (7312) Queuing response >Thread-16::DEBUG::2014-08-11 15:35:10,284::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000437274 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:10,285::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:10,285::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:10,286::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:10,286::__init__::232::IOProcess::(_processLogs) (7313) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:10,287::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7313) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:10,288::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:10,289::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:10,290::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:10,351::__init__::232::IOProcess::(_processLogs) (7314) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:10,352::__init__::232::IOProcess::(_processLogs) (7314) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:10,352::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:10,353::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:10,353::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:10,354::__init__::232::IOProcess::(_processLogs) (7315) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:10,354::__init__::232::IOProcess::(_processLogs) (7315) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:10,355::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:10,356::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:10,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:10,359::__init__::232::IOProcess::(_processLogs) (7316) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:10,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:10,361::__init__::232::IOProcess::(_processLogs) (7316) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:10,363::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:10,364::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:10,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:10,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013569 s, 75.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:35:11,671::task::595::Storage.TaskManager.Task::(_updateState) Task=`6614c036-fa74-45e3-a827-8b6b93d95b36`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:11,672::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:35:11,672::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000437274', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:11,672::task::1191::Storage.TaskManager.Task::(prepare) Task=`6614c036-fa74-45e3-a827-8b6b93d95b36`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000437274', 'lastCheck': '1.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:11,672::task::595::Storage.TaskManager.Task::(_updateState) Task=`6614c036-fa74-45e3-a827-8b6b93d95b36`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:11,673::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:11,673::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:11,673::task::993::Storage.TaskManager.Task::(_decref) Task=`6614c036-fa74-45e3-a827-8b6b93d95b36`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:35:11,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:12,978::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:13,011::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131717 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:15,034::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:15,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176562 s, 58.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:15,371::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:15,373::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:35:15,426::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:35:15,454::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed3e7248-67e1-4dc6-86fd-7a07141b7075`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:15,456::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:15,456::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:35:15,470::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:15,470::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed3e7248-67e1-4dc6-86fd-7a07141b7075`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:15,470::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed3e7248-67e1-4dc6-86fd-7a07141b7075`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:15,471::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:15,471::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:15,471::task::993::Storage.TaskManager.Task::(_decref) Task=`ed3e7248-67e1-4dc6-86fd-7a07141b7075`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:35:15,483::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:35:15,484::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d769fc7-e3ca-43f8-a6ea-13517d601e2a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:35:15,484::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:35:15,485::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cf919eca-a09a-45e1-ad2b-13c50709edb2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:35:15,485::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:35:15,485::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:35:15,485::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cf919eca-a09a-45e1-ad2b-13c50709edb2`::Granted request >Thread-34::DEBUG::2014-08-11 15:35:15,486::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0d769fc7-e3ca-43f8-a6ea-13517d601e2a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:35:15,486::task::993::Storage.TaskManager.Task::(_decref) Task=`0d769fc7-e3ca-43f8-a6ea-13517d601e2a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:35:15,506::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:15,506::task::1191::Storage.TaskManager.Task::(prepare) Task=`0d769fc7-e3ca-43f8-a6ea-13517d601e2a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:15,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`0d769fc7-e3ca-43f8-a6ea-13517d601e2a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:35:15,507::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:35:15,507::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:35:15,508::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:35:15,508::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:35:15,508::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:35:15,508::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:35:15,509::task::993::Storage.TaskManager.Task::(_decref) Task=`0d769fc7-e3ca-43f8-a6ea-13517d601e2a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:35:17,093::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:17,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132188 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:19,149::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:19,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133279 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:20,358::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:20,359::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:20,359::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:20,359::__init__::232::IOProcess::(_processLogs) (7317) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:20,359::__init__::232::IOProcess::(_processLogs) (7317) Queuing response >Thread-16::DEBUG::2014-08-11 15:35:20,360::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:35:20,379::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420076 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:20,380::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:20,452::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:20,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:20,454::__init__::232::IOProcess::(_processLogs) (7318) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:20,454::__init__::232::IOProcess::(_processLogs) (7318) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:20,454::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:20,455::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:20,455::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:20,455::__init__::232::IOProcess::(_processLogs) (7319) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:20,455::__init__::232::IOProcess::(_processLogs) (7319) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:20,456::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:20,457::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:20,457::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:20,457::__init__::232::IOProcess::(_processLogs) (7320) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:20,458::__init__::232::IOProcess::(_processLogs) (7320) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:20,458::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:20,459::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:20,459::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:20,460::__init__::232::IOProcess::(_processLogs) (7321) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:20,460::__init__::232::IOProcess::(_processLogs) (7321) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:20,464::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:20,465::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:20,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:21,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:21,237::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132537 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:23,259::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:23,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131785 s, 77.7 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:35:24,314::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b0331ca-c1f9-4b1b-975b-7f4ba88e5ce7`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:35:24,315::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:35:24,315::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:24,316::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:24,316::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:24,316::__init__::232::IOProcess::(_processLogs) (7322) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:35:24,317::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7322) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:24,318::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:24,319::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:35:24,319::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:35:24,320::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:35:24,321::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b0331ca-c1f9-4b1b-975b-7f4ba88e5ce7`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:35:24,321::__init__::232::IOProcess::(_processLogs) (7323) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:35:24,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b0331ca-c1f9-4b1b-975b-7f4ba88e5ce7`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:35:24,323::__init__::232::IOProcess::(_processLogs) (7323) Queuing response >Thread-122::DEBUG::2014-08-11 15:35:24,323::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:35:24,324::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:35:24,324::task::993::Storage.TaskManager.Task::(_decref) Task=`9b0331ca-c1f9-4b1b-975b-7f4ba88e5ce7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:35:25,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:25,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013309 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:25,469::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:25,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:25,472::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:35:25,523::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:35:25,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`76abe01c-f033-48ca-ae2f-952397b4a7e2`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:25,524::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:35:25,537::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:25,537::task::1191::Storage.TaskManager.Task::(prepare) Task=`76abe01c-f033-48ca-ae2f-952397b4a7e2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:25,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`76abe01c-f033-48ca-ae2f-952397b4a7e2`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:25,538::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:25,538::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:25,538::task::993::Storage.TaskManager.Task::(_decref) Task=`76abe01c-f033-48ca-ae2f-952397b4a7e2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:35:25,550::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:35:25,550::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e0e4149-9970-49aa-9fc9-7f0cd69984b2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:35:25,551::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:35:25,551::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dc1d42bb-5e08-48dc-b5b1-3393e084cb04`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:35:25,552::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:35:25,552::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:35:25,552::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dc1d42bb-5e08-48dc-b5b1-3393e084cb04`::Granted request >Thread-34::DEBUG::2014-08-11 15:35:25,553::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8e0e4149-9970-49aa-9fc9-7f0cd69984b2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:35:25,553::task::993::Storage.TaskManager.Task::(_decref) Task=`8e0e4149-9970-49aa-9fc9-7f0cd69984b2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:35:25,573::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:25,573::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e0e4149-9970-49aa-9fc9-7f0cd69984b2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:25,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e0e4149-9970-49aa-9fc9-7f0cd69984b2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:35:25,574::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:35:25,574::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:35:25,574::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:35:25,575::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:35:25,575::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:35:25,575::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:35:25,575::task::993::Storage.TaskManager.Task::(_decref) Task=`8e0e4149-9970-49aa-9fc9-7f0cd69984b2`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:35:26,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`06f4394e-976f-42e0-9954-43fda1228e6b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:26,782::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:35:26,782::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420076', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:26,783::task::1191::Storage.TaskManager.Task::(prepare) Task=`06f4394e-976f-42e0-9954-43fda1228e6b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420076', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:26,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`06f4394e-976f-42e0-9954-43fda1228e6b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:26,783::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:26,783::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:26,783::task::993::Storage.TaskManager.Task::(_decref) Task=`06f4394e-976f-42e0-9954-43fda1228e6b`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:35:26,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:27,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:27,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132486 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:29,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:29,460::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134002 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:30,461::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:30,461::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:30,462::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:30,462::__init__::232::IOProcess::(_processLogs) (7324) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:30,462::__init__::232::IOProcess::(_processLogs) (7324) Queuing response >Thread-16::DEBUG::2014-08-11 15:35:30,463::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:30,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:35:30,483::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000453724 s, 1.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:30,485::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:30,553::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:30,554::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:30,556::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:30,557::__init__::232::IOProcess::(_processLogs) (7325) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:30,558::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:30,559::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7325) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:30,561::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:30,563::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:30,564::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:30,564::__init__::232::IOProcess::(_processLogs) (7326) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:30,564::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7326) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:30,565::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:30,565::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:30,566::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:30,566::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7327) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:30,567::__init__::232::IOProcess::(_processLogs) (7327) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:30,567::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:30,568::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:30,568::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:30,568::__init__::232::IOProcess::(_processLogs) (7328) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:30,569::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7328) Queuing response >Dummy-33::DEBUG::2014-08-11 15:35:31,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:31,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133163 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:33,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:33,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131884 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:35,567::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:35,568::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:35:35,588::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Dummy-33::DEBUG::2014-08-11 15:35:35,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:35,673::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:35:35,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1e92ba7-ae8e-4ca3-95f9-985dfc27089b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:35,675::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 15:35:35,687::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135395 s, 75.6 MB/s\n'; <rc> = 0 >Thread-115::INFO::2014-08-11 15:35:35,704::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:35,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`e1e92ba7-ae8e-4ca3-95f9-985dfc27089b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:35,706::task::595::Storage.TaskManager.Task::(_updateState) Task=`e1e92ba7-ae8e-4ca3-95f9-985dfc27089b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:35,707::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:35,708::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:35,710::task::993::Storage.TaskManager.Task::(_decref) Task=`e1e92ba7-ae8e-4ca3-95f9-985dfc27089b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:35:35,724::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:35:35,725::task::595::Storage.TaskManager.Task::(_updateState) Task=`574a34e9-5d30-40e6-81de-1ee7aa544cd2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:35:35,725::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:35:35,726::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1ceb6dae-57af-407f-9da9-edf6291e8ed7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:35:35,726::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:35:35,726::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:35:35,727::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1ceb6dae-57af-407f-9da9-edf6291e8ed7`::Granted request >Thread-34::DEBUG::2014-08-11 15:35:35,727::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`574a34e9-5d30-40e6-81de-1ee7aa544cd2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:35:35,727::task::993::Storage.TaskManager.Task::(_decref) Task=`574a34e9-5d30-40e6-81de-1ee7aa544cd2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:35:35,747::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:35,747::task::1191::Storage.TaskManager.Task::(prepare) Task=`574a34e9-5d30-40e6-81de-1ee7aa544cd2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:35,748::task::595::Storage.TaskManager.Task::(_updateState) Task=`574a34e9-5d30-40e6-81de-1ee7aa544cd2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:35:35,748::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:35:35,748::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:35:35,748::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:35:35,749::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:35:35,749::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:35:35,749::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:35:35,750::task::993::Storage.TaskManager.Task::(_decref) Task=`574a34e9-5d30-40e6-81de-1ee7aa544cd2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:35:37,718::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:37,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131166 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:39,773::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:39,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132005 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:40,570::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:40,571::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:40,652::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:40,653::__init__::232::IOProcess::(_processLogs) (7329) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:40,653::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7329) Queuing response >Thread-16::DEBUG::2014-08-11 15:35:40,654::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:35:40,673::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415929 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:40,675::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:40,675::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:40,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:40,675::__init__::232::IOProcess::(_processLogs) (7330) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:40,676::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7330) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:40,677::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:40,678::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:40,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:40,679::__init__::232::IOProcess::(_processLogs) (7331) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:40,679::__init__::232::IOProcess::(_processLogs) (7331) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:40,682::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:40,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:40,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:40,686::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:40,687::__init__::232::IOProcess::(_processLogs) (7332) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:40,688::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:40,690::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7332) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:40,692::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:40,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:40,694::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:40,695::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:40,696::__init__::232::IOProcess::(_processLogs) (7333) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:40,696::__init__::232::IOProcess::(_processLogs) (7333) Queuing response >Dummy-33::DEBUG::2014-08-11 15:35:41,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:41,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132505 s, 77.3 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:35:41,906::task::595::Storage.TaskManager.Task::(_updateState) Task=`62716813-5d56-4049-97d5-ab0d32bdccb7`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:41,906::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:35:41,907::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415929', 'lastCheck': '1.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:41,908::task::1191::Storage.TaskManager.Task::(prepare) Task=`62716813-5d56-4049-97d5-ab0d32bdccb7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415929', 'lastCheck': '1.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:41,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`62716813-5d56-4049-97d5-ab0d32bdccb7`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:41,908::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:41,908::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:41,909::task::993::Storage.TaskManager.Task::(_decref) Task=`62716813-5d56-4049-97d5-ab0d32bdccb7`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:35:41,930::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:43,889::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:43,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133487 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:45,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:45,703::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:45,753::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:35:45,768::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:35:45,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`a97e6337-7186-4c79-9fad-a60a330919cb`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:45,769::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:35:45,782::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:45,782::task::1191::Storage.TaskManager.Task::(prepare) Task=`a97e6337-7186-4c79-9fad-a60a330919cb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:45,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`a97e6337-7186-4c79-9fad-a60a330919cb`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:45,783::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:45,783::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:45,783::task::993::Storage.TaskManager.Task::(_decref) Task=`a97e6337-7186-4c79-9fad-a60a330919cb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:35:45,795::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:35:45,795::task::595::Storage.TaskManager.Task::(_updateState) Task=`27046737-5e33-4fd9-916a-acbf134d6ed3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:35:45,796::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:35:45,797::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`44bfd599-54d0-462a-b6e9-83aa991659bf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:35:45,797::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:35:45,797::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:35:45,797::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`44bfd599-54d0-462a-b6e9-83aa991659bf`::Granted request >Thread-34::DEBUG::2014-08-11 15:35:45,798::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`27046737-5e33-4fd9-916a-acbf134d6ed3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:35:45,798::task::993::Storage.TaskManager.Task::(_decref) Task=`27046737-5e33-4fd9-916a-acbf134d6ed3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:35:45,818::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:45,818::task::1191::Storage.TaskManager.Task::(prepare) Task=`27046737-5e33-4fd9-916a-acbf134d6ed3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:45,818::task::595::Storage.TaskManager.Task::(_updateState) Task=`27046737-5e33-4fd9-916a-acbf134d6ed3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:35:45,819::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:35:45,819::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:35:45,819::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:35:45,820::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:35:45,820::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:35:45,820::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:35:45,820::task::993::Storage.TaskManager.Task::(_decref) Task=`27046737-5e33-4fd9-916a-acbf134d6ed3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:35:45,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:45,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218899 s, 46.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:48,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:48,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131967 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:50,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:50,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132688 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:50,699::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:50,699::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:50,699::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:50,700::__init__::232::IOProcess::(_processLogs) (7334) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:50,700::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7334) Queuing response >Thread-16::DEBUG::2014-08-11 15:35:50,702::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:35:50,720::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420293 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:35:50,722::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:50,753::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:50,754::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:50,754::__init__::232::IOProcess::(_processLogs) (7335) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:35:50,755::__init__::232::IOProcess::(_processLogs) (7335) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:50,755::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:50,755::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:50,756::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:50,756::__init__::232::IOProcess::(_processLogs) (7336) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:50,756::__init__::232::IOProcess::(_processLogs) (7336) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:50,757::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:50,757::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:50,758::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:35:50,759::__init__::232::IOProcess::(_processLogs) (7337) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:50,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:50,761::__init__::232::IOProcess::(_processLogs) (7337) Queuing response >Thread-14::DEBUG::2014-08-11 15:35:50,763::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:35:50,765::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:35:50,766::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:50,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:35:50,767::__init__::232::IOProcess::(_processLogs) (7338) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:35:50,769::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7338) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:50,772::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:52,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:52,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132063 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:35:54,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:54,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132814 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:55,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:55,780::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:35:55,833::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-115::DEBUG::2014-08-11 15:35:55,855::task::595::Storage.TaskManager.Task::(_updateState) Task=`8991b0b4-2b64-4405-ad81-445f77c235fc`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:55,857::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:35:55,857::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:35:55,871::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:55,871::task::1191::Storage.TaskManager.Task::(prepare) Task=`8991b0b4-2b64-4405-ad81-445f77c235fc`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:35:55,871::task::595::Storage.TaskManager.Task::(_updateState) Task=`8991b0b4-2b64-4405-ad81-445f77c235fc`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:55,872::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:55,872::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:55,872::task::993::Storage.TaskManager.Task::(_decref) Task=`8991b0b4-2b64-4405-ad81-445f77c235fc`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:35:55,883::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 15:35:55,883::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ec483b5-d369-4377-ba86-cf08908e9ad4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:35:55,884::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:35:55,885::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e848c2b7-256c-4f92-86e6-9fd4c8738195`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:35:55,885::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:35:55,885::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:35:55,885::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e848c2b7-256c-4f92-86e6-9fd4c8738195`::Granted request >Thread-34::DEBUG::2014-08-11 15:35:55,886::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5ec483b5-d369-4377-ba86-cf08908e9ad4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:35:55,886::task::993::Storage.TaskManager.Task::(_decref) Task=`5ec483b5-d369-4377-ba86-cf08908e9ad4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:35:55,906::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:55,906::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ec483b5-d369-4377-ba86-cf08908e9ad4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:35:55,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ec483b5-d369-4377-ba86-cf08908e9ad4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:35:55,907::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:35:55,907::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:35:55,907::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:35:55,908::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:35:55,908::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:35:55,908::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:35:55,908::task::993::Storage.TaskManager.Task::(_decref) Task=`5ec483b5-d369-4377-ba86-cf08908e9ad4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:35:56,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:56,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134775 s, 76.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:35:57,027::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b8a1b08-286a-4258-8aac-0d03e9aeb2e4`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:35:57,027::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:35:57,028::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420293', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:57,028::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b8a1b08-286a-4258-8aac-0d03e9aeb2e4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420293', 'lastCheck': '6.3', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:35:57,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b8a1b08-286a-4258-8aac-0d03e9aeb2e4`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:35:57,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:35:57,029::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:35:57,029::task::993::Storage.TaskManager.Task::(_decref) Task=`8b8a1b08-286a-4258-8aac-0d03e9aeb2e4`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:35:57,052::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:35:58,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:35:58,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132386 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:00,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:00,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134893 s, 75.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:00,772::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:00,772::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:00,772::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:00,773::__init__::232::IOProcess::(_processLogs) (7339) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:00,773::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7339) Queuing response >Thread-16::DEBUG::2014-08-11 15:36:00,774::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:36:00,792::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042131 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:00,853::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:00,854::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:00,854::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:00,854::__init__::232::IOProcess::(_processLogs) (7340) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:00,855::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7340) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:00,856::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:00,857::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:00,858::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:00,858::__init__::232::IOProcess::(_processLogs) (7341) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:00,858::__init__::232::IOProcess::(_processLogs) (7341) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:00,859::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:00,859::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:00,860::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:00,860::__init__::232::IOProcess::(_processLogs) (7342) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:00,860::__init__::232::IOProcess::(_processLogs) (7342) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:00,861::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:00,862::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:00,862::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:00,863::__init__::232::IOProcess::(_processLogs) (7343) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:00,863::__init__::232::IOProcess::(_processLogs) (7343) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:00,866::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:00,867::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:00,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:36:02,412::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:02,462::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0279252 s, 36.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:04,679::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:04,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132874 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:05,872::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:05,873::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:36:05,920::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:36:05,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae7e0f45-a247-494d-9b68-ef17d62015ca`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:05,957::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:05,957::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:36:05,971::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:05,971::task::1191::Storage.TaskManager.Task::(prepare) Task=`ae7e0f45-a247-494d-9b68-ef17d62015ca`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:05,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae7e0f45-a247-494d-9b68-ef17d62015ca`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:05,972::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:05,972::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:05,972::task::993::Storage.TaskManager.Task::(_decref) Task=`ae7e0f45-a247-494d-9b68-ef17d62015ca`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:36:05,983::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:36:05,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e313626-0b31-434f-8abe-de885c04c4b4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:36:05,984::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:36:05,985::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cbcf5b2f-d409-422f-ad93-1ae76853bce7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:36:05,985::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:36:05,986::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:36:05,986::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cbcf5b2f-d409-422f-ad93-1ae76853bce7`::Granted request >Thread-34::DEBUG::2014-08-11 15:36:05,987::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0e313626-0b31-434f-8abe-de885c04c4b4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:36:05,987::task::993::Storage.TaskManager.Task::(_decref) Task=`0e313626-0b31-434f-8abe-de885c04c4b4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:36:06,007::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:06,007::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e313626-0b31-434f-8abe-de885c04c4b4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:06,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e313626-0b31-434f-8abe-de885c04c4b4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:36:06,008::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:36:06,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:36:06,008::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:36:06,009::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:36:06,009::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:36:06,009::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:36:06,009::task::993::Storage.TaskManager.Task::(_decref) Task=`0e313626-0b31-434f-8abe-de885c04c4b4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:36:06,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:06,767::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131141 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:08,789::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:08,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132054 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:10,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:36:10,865::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:10,865::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:10,866::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:10,866::__init__::232::IOProcess::(_processLogs) (7344) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:10,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7344) Queuing response >Thread-16::DEBUG::2014-08-11 15:36:10,874::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:10,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0336709 s, 30.4 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:36:10,897::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00439075 s, 104 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:10,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:10,960::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:10,961::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:10,963::__init__::232::IOProcess::(_processLogs) (7345) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:10,965::__init__::232::IOProcess::(_processLogs) (7345) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:10,967::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:10,972::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:10,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:10,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:10,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:10,981::__init__::232::IOProcess::(_processLogs) (7346) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:10,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:10,983::__init__::232::IOProcess::(_processLogs) (7346) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:10,987::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:10,989::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:10,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:10,992::__init__::232::IOProcess::(_processLogs) (7347) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:10,993::__init__::232::IOProcess::(_processLogs) (7347) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:10,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:10,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:10,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:10,999::__init__::232::IOProcess::(_processLogs) (7348) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:11,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7348) Queuing response >Thread-115::DEBUG::2014-08-11 15:36:12,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`82f285ca-d86d-4966-8083-9b474bdf7d9e`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:12,151::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:36:12,151::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00439075', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:12,151::task::1191::Storage.TaskManager.Task::(prepare) Task=`82f285ca-d86d-4966-8083-9b474bdf7d9e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00439075', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:12,151::task::595::Storage.TaskManager.Task::(_updateState) Task=`82f285ca-d86d-4966-8083-9b474bdf7d9e`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:12,152::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:12,152::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:12,152::task::993::Storage.TaskManager.Task::(_decref) Task=`82f285ca-d86d-4966-8083-9b474bdf7d9e`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:36:12,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:36:13,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:13,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0243887 s, 42.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:15,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:15,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131647 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:15,993::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:15,994::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:36:16,026::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:16,055::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:36:16,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`cace8e78-03a3-4826-8ba5-4d17e0c4218b`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:16,058::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:36:16,072::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:16,072::task::1191::Storage.TaskManager.Task::(prepare) Task=`cace8e78-03a3-4826-8ba5-4d17e0c4218b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:16,073::task::595::Storage.TaskManager.Task::(_updateState) Task=`cace8e78-03a3-4826-8ba5-4d17e0c4218b`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:16,073::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:16,073::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:16,073::task::993::Storage.TaskManager.Task::(_decref) Task=`cace8e78-03a3-4826-8ba5-4d17e0c4218b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:36:16,084::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:36:16,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`2689e0c2-260c-4fd0-a9a2-cbee1d7c93a7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:36:16,085::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:36:16,085::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d55a4a69-6ccc-41f6-a09c-6bf2f06b0673`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:36:16,086::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:36:16,088::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:36:16,088::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d55a4a69-6ccc-41f6-a09c-6bf2f06b0673`::Granted request >Thread-34::DEBUG::2014-08-11 15:36:16,089::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2689e0c2-260c-4fd0-a9a2-cbee1d7c93a7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:36:16,089::task::993::Storage.TaskManager.Task::(_decref) Task=`2689e0c2-260c-4fd0-a9a2-cbee1d7c93a7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:36:16,109::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:16,109::task::1191::Storage.TaskManager.Task::(prepare) Task=`2689e0c2-260c-4fd0-a9a2-cbee1d7c93a7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:16,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`2689e0c2-260c-4fd0-a9a2-cbee1d7c93a7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:36:16,110::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:36:16,110::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:36:16,110::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:36:16,111::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:36:16,111::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:36:16,111::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:36:16,111::task::993::Storage.TaskManager.Task::(_decref) Task=`2689e0c2-260c-4fd0-a9a2-cbee1d7c93a7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:36:17,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:17,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131811 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:19,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:19,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133964 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:21,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:21,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:21,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:36:21,055::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:36:21,055::__init__::232::IOProcess::(_processLogs) (7349) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:21,057::__init__::232::IOProcess::(_processLogs) (7349) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:21,077::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:36:21,078::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000762318 s, 597 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:21,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:21,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:21,084::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:21,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:21,086::__init__::232::IOProcess::(_processLogs) (7350) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:21,088::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7350) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:21,090::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:21,091::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:21,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:21,093::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:21,094::__init__::232::IOProcess::(_processLogs) (7351) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:21,095::__init__::232::IOProcess::(_processLogs) (7351) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:21,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:21,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:21,097::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:21,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7352) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:21,098::__init__::232::IOProcess::(_processLogs) (7352) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:21,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:21,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:21,100::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:21,101::__init__::232::IOProcess::(_processLogs) (7353) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:21,101::__init__::232::IOProcess::(_processLogs) (7353) Queuing response >Dummy-33::DEBUG::2014-08-11 15:36:21,355::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:21,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133723 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:23,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:23,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131261 s, 78.0 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:36:24,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`af09b5a9-1a1e-4cca-86d7-dbd36787ec71`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:36:24,428::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:36:24,429::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:24,429::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:24,430::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:24,430::__init__::232::IOProcess::(_processLogs) (7354) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:36:24,430::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7354) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:24,431::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:24,432::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:36:24,432::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:36:24,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:36:24,434::task::1191::Storage.TaskManager.Task::(prepare) Task=`af09b5a9-1a1e-4cca-86d7-dbd36787ec71`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:36:24,435::__init__::232::IOProcess::(_processLogs) (7355) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:36:24,435::task::595::Storage.TaskManager.Task::(_updateState) Task=`af09b5a9-1a1e-4cca-86d7-dbd36787ec71`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:36:24,436::__init__::232::IOProcess::(_processLogs) (7355) Queuing response >Thread-122::DEBUG::2014-08-11 15:36:24,437::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:36:24,437::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:36:24,437::task::993::Storage.TaskManager.Task::(_decref) Task=`af09b5a9-1a1e-4cca-86d7-dbd36787ec71`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:36:25,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:25,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133085 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:26,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:26,101::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:36:26,126::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:36:26,156::task::595::Storage.TaskManager.Task::(_updateState) Task=`d215351d-ac0e-48f4-badc-814522aa40af`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:26,158::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:26,158::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:36:26,172::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:26,172::task::1191::Storage.TaskManager.Task::(prepare) Task=`d215351d-ac0e-48f4-badc-814522aa40af`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:26,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`d215351d-ac0e-48f4-badc-814522aa40af`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:26,173::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:26,173::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:26,173::task::993::Storage.TaskManager.Task::(_decref) Task=`d215351d-ac0e-48f4-badc-814522aa40af`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:36:26,184::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:36:26,184::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ab673a3-d4c2-40f2-9f23-74632f899525`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:36:26,184::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:36:26,185::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`47234227-aecf-4ff0-9b24-f5f31a9263f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:36:26,186::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:36:26,186::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:36:26,186::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`47234227-aecf-4ff0-9b24-f5f31a9263f8`::Granted request >Thread-34::DEBUG::2014-08-11 15:36:26,187::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1ab673a3-d4c2-40f2-9f23-74632f899525`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:36:26,187::task::993::Storage.TaskManager.Task::(_decref) Task=`1ab673a3-d4c2-40f2-9f23-74632f899525`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:36:26,207::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:26,207::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ab673a3-d4c2-40f2-9f23-74632f899525`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952543232', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:26,207::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ab673a3-d4c2-40f2-9f23-74632f899525`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:36:26,208::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:36:26,208::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:36:26,208::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:36:26,208::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:36:26,209::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:36:26,209::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:36:26,209::task::993::Storage.TaskManager.Task::(_decref) Task=`1ab673a3-d4c2-40f2-9f23-74632f899525`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:36:27,269::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab8b43f1-5ba4-4da4-9dbe-5a007d532b99`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:27,269::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:36:27,269::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000762318', 'lastCheck': '6.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:27,270::task::1191::Storage.TaskManager.Task::(prepare) Task=`ab8b43f1-5ba4-4da4-9dbe-5a007d532b99`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000762318', 'lastCheck': '6.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:27,270::task::595::Storage.TaskManager.Task::(_updateState) Task=`ab8b43f1-5ba4-4da4-9dbe-5a007d532b99`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:27,270::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:27,270::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:27,271::task::993::Storage.TaskManager.Task::(_decref) Task=`ab8b43f1-5ba4-4da4-9dbe-5a007d532b99`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:36:27,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:36:27,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:27,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134965 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:29,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:29,610::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133563 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:31,101::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:31,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:36:31,104::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:36:31,154::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:31,173::__init__::232::IOProcess::(_processLogs) (7356) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:31,175::__init__::232::IOProcess::(_processLogs) (7356) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:31,176::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:36:31,178::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000970576 s, 469 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:31,181::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:31,182::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:31,184::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:31,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:31,186::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:31,188::__init__::232::IOProcess::(_processLogs) (7357) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:31,190::__init__::232::IOProcess::(_processLogs) (7357) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:31,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:31,191::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:31,191::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:31,191::__init__::232::IOProcess::(_processLogs) (7358) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:31,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7358) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:31,193::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:31,193::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:31,194::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:31,194::__init__::232::IOProcess::(_processLogs) (7359) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:31,195::__init__::232::IOProcess::(_processLogs) (7359) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:31,195::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:31,195::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:31,196::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:31,196::__init__::232::IOProcess::(_processLogs) (7360) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:31,197::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7360) Queuing response >Dummy-33::DEBUG::2014-08-11 15:36:31,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:31,668::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132973 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:33,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:33,724::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131797 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:35,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:35,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132353 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:36,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:36,194::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:36:36,221::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:36:36,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ec66f81-3efc-422c-9393-a6a9efd0c06d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:36,258::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:36,258::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:36:36,273::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:36,273::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ec66f81-3efc-422c-9393-a6a9efd0c06d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:36,273::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ec66f81-3efc-422c-9393-a6a9efd0c06d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:36,273::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:36,274::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:36,274::task::993::Storage.TaskManager.Task::(_decref) Task=`0ec66f81-3efc-422c-9393-a6a9efd0c06d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:36:36,284::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:36:36,284::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d4556ae-ed62-4a40-aad7-d4b3f235870c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:36:36,285::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:36:36,286::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2293cc4f-dfe9-4472-99c0-5ac64c1510f9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:36:36,286::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:36:36,286::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:36:36,286::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2293cc4f-dfe9-4472-99c0-5ac64c1510f9`::Granted request >Thread-34::DEBUG::2014-08-11 15:36:36,287::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5d4556ae-ed62-4a40-aad7-d4b3f235870c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:36:36,287::task::993::Storage.TaskManager.Task::(_decref) Task=`5d4556ae-ed62-4a40-aad7-d4b3f235870c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:36:36,307::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952477696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:36,307::task::1191::Storage.TaskManager.Task::(prepare) Task=`5d4556ae-ed62-4a40-aad7-d4b3f235870c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952477696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:36,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d4556ae-ed62-4a40-aad7-d4b3f235870c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:36:36,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:36:36,308::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:36:36,308::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:36:36,309::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:36:36,309::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:36:36,309::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:36:36,309::task::993::Storage.TaskManager.Task::(_decref) Task=`5d4556ae-ed62-4a40-aad7-d4b3f235870c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:36:37,801::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:37,834::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133103 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:39,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:39,890::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131969 s, 77.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:41,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:41,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:36:41,200::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:36:41,254::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:41,273::__init__::232::IOProcess::(_processLogs) (7361) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:41,275::__init__::232::IOProcess::(_processLogs) (7361) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:41,275::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:36:41,278::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000740127 s, 615 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:41,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:41,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:41,283::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:36:41,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:41,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:41,287::__init__::232::IOProcess::(_processLogs) (7362) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:41,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7362) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:41,289::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:41,289::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:41,290::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:41,290::__init__::232::IOProcess::(_processLogs) (7363) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:41,290::__init__::232::IOProcess::(_processLogs) (7363) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:41,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:41,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:41,291::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:41,291::__init__::232::IOProcess::(_processLogs) (7364) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:41,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7364) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:41,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:41,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:41,295::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:41,295::__init__::232::IOProcess::(_processLogs) (7365) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:41,295::__init__::232::IOProcess::(_processLogs) (7365) Queuing response >Dummy-33::DEBUG::2014-08-11 15:36:41,912::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:41,945::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131214 s, 78.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:36:42,384::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdd1e520-8f0f-4e09-9bb7-017bf104e042`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:42,385::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:36:42,385::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000740127', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:42,385::task::1191::Storage.TaskManager.Task::(prepare) Task=`fdd1e520-8f0f-4e09-9bb7-017bf104e042`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000740127', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:42,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`fdd1e520-8f0f-4e09-9bb7-017bf104e042`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:42,386::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:42,386::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:42,386::task::993::Storage.TaskManager.Task::(_decref) Task=`fdd1e520-8f0f-4e09-9bb7-017bf104e042`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:36:42,404::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:36:43,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:44,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132389 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:46,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:46,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135665 s, 75.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:46,293::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:46,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:46,296::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:36:46,323::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-115::DEBUG::2014-08-11 15:36:46,324::task::595::Storage.TaskManager.Task::(_updateState) Task=`0981867a-9c86-47b0-a74a-464ac848e540`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:46,324::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:36:46,344::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:46,345::task::1191::Storage.TaskManager.Task::(prepare) Task=`0981867a-9c86-47b0-a74a-464ac848e540`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:46,345::task::595::Storage.TaskManager.Task::(_updateState) Task=`0981867a-9c86-47b0-a74a-464ac848e540`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:46,345::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:46,345::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:46,346::task::993::Storage.TaskManager.Task::(_decref) Task=`0981867a-9c86-47b0-a74a-464ac848e540`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:36:46,357::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 15:36:46,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb829713-6473-4c00-97b3-f3cc4e4f17bf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:36:46,358::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:36:46,359::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`211efed2-7099-4203-858d-3a33056b1a8c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:36:46,359::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:36:46,359::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:36:46,360::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`211efed2-7099-4203-858d-3a33056b1a8c`::Granted request >Thread-34::DEBUG::2014-08-11 15:36:46,360::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bb829713-6473-4c00-97b3-f3cc4e4f17bf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:36:46,361::task::993::Storage.TaskManager.Task::(_decref) Task=`bb829713-6473-4c00-97b3-f3cc4e4f17bf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:36:46,383::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:46,383::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb829713-6473-4c00-97b3-f3cc4e4f17bf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:46,384::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb829713-6473-4c00-97b3-f3cc4e4f17bf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:36:46,384::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:36:46,384::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:36:46,384::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:36:46,385::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:36:46,385::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:36:46,385::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:36:46,385::task::993::Storage.TaskManager.Task::(_decref) Task=`bb829713-6473-4c00-97b3-f3cc4e4f17bf`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:36:48,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:48,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132506 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:50,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:50,171::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155925 s, 65.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:51,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:51,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:36:51,298::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:36:51,298::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:51,317::__init__::232::IOProcess::(_processLogs) (7366) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:51,317::__init__::232::IOProcess::(_processLogs) (7366) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:51,319::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:51,320::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:51,322::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:36:51,325::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000504128 s, 903 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:36:51,326::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:51,326::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:51,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:51,328::__init__::232::IOProcess::(_processLogs) (7367) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:36:51,328::__init__::232::IOProcess::(_processLogs) (7367) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:51,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:51,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:51,330::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:51,330::__init__::232::IOProcess::(_processLogs) (7368) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:51,330::__init__::232::IOProcess::(_processLogs) (7368) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:51,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:51,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:51,332::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:51,332::__init__::232::IOProcess::(_processLogs) (7369) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:51,332::__init__::232::IOProcess::(_processLogs) (7369) Queuing response >Thread-14::DEBUG::2014-08-11 15:36:51,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:36:51,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:36:51,334::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:36:51,335::__init__::232::IOProcess::(_processLogs) (7370) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:36:51,335::__init__::232::IOProcess::(_processLogs) (7370) Queuing response >Dummy-33::DEBUG::2014-08-11 15:36:52,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:52,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187957 s, 54.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:54,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:54,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131636 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:36:56,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:56,332::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:56,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:36:56,335::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:36:56,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133382 s, 76.8 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:36:56,398::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:36:56,399::task::595::Storage.TaskManager.Task::(_updateState) Task=`812896db-3e97-461b-a04f-8d86daaefd39`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:56,399::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:36:56,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:56,412::task::1191::Storage.TaskManager.Task::(prepare) Task=`812896db-3e97-461b-a04f-8d86daaefd39`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:36:56,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`812896db-3e97-461b-a04f-8d86daaefd39`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:56,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:56,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:56,413::task::993::Storage.TaskManager.Task::(_decref) Task=`812896db-3e97-461b-a04f-8d86daaefd39`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:36:56,423::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:36:56,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`7102c78f-696e-4277-ae91-b6a3659da7bf`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:36:56,424::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:36:56,425::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7241cba-e565-4192-ad7e-9d7398760b60`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:36:56,425::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:36:56,426::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:36:56,426::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c7241cba-e565-4192-ad7e-9d7398760b60`::Granted request >Thread-34::DEBUG::2014-08-11 15:36:56,427::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7102c78f-696e-4277-ae91-b6a3659da7bf`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:36:56,427::task::993::Storage.TaskManager.Task::(_decref) Task=`7102c78f-696e-4277-ae91-b6a3659da7bf`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:36:56,447::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952477696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:56,447::task::1191::Storage.TaskManager.Task::(prepare) Task=`7102c78f-696e-4277-ae91-b6a3659da7bf`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952477696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:36:56,447::task::595::Storage.TaskManager.Task::(_updateState) Task=`7102c78f-696e-4277-ae91-b6a3659da7bf`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:36:56,448::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:36:56,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:36:56,448::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:36:56,449::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:36:56,449::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:36:56,449::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:36:56,449::task::993::Storage.TaskManager.Task::(_decref) Task=`7102c78f-696e-4277-ae91-b6a3659da7bf`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:36:57,496::task::595::Storage.TaskManager.Task::(_updateState) Task=`0530cee4-3d4b-4ed8-883e-71bcb5fd536d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:36:57,496::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:36:57,497::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000504128', 'lastCheck': '6.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:57,497::task::1191::Storage.TaskManager.Task::(prepare) Task=`0530cee4-3d4b-4ed8-883e-71bcb5fd536d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000504128', 'lastCheck': '6.2', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:36:57,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`0530cee4-3d4b-4ed8-883e-71bcb5fd536d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:36:57,497::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:36:57,497::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:36:57,498::task::993::Storage.TaskManager.Task::(_decref) Task=`0530cee4-3d4b-4ed8-883e-71bcb5fd536d`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:36:57,519::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:36:58,369::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:36:58,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133721 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:00,426::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:00,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133811 s, 76.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:01,335::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:01,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:01,342::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:01,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:01,357::__init__::232::IOProcess::(_processLogs) (7371) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:01,359::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7371) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:01,361::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:01,362::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:37:01,384::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000802859 s, 567 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:01,386::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:01,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:01,391::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:01,391::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:01,391::__init__::232::IOProcess::(_processLogs) (7372) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:01,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7372) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:01,393::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:01,394::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:01,394::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:01,394::__init__::232::IOProcess::(_processLogs) (7373) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:01,395::__init__::232::IOProcess::(_processLogs) (7373) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:01,395::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:01,396::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:01,396::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:01,396::__init__::232::IOProcess::(_processLogs) (7374) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:01,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7374) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:01,398::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:01,398::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:01,398::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:01,398::__init__::232::IOProcess::(_processLogs) (7375) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:01,399::__init__::232::IOProcess::(_processLogs) (7375) Queuing response >Dummy-33::DEBUG::2014-08-11 15:37:02,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:02,570::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344983 s, 29.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:04,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:04,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133163 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:06,397::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:06,398::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:06,399::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:37:06,462::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:37:06,463::task::595::Storage.TaskManager.Task::(_updateState) Task=`62310b14-9e13-4418-afe7-787f5660e865`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:06,463::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:37:06,476::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:06,476::task::1191::Storage.TaskManager.Task::(prepare) Task=`62310b14-9e13-4418-afe7-787f5660e865`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:06,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`62310b14-9e13-4418-afe7-787f5660e865`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:06,477::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:06,477::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:06,477::task::993::Storage.TaskManager.Task::(_decref) Task=`62310b14-9e13-4418-afe7-787f5660e865`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:37:06,487::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:37:06,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`91beeccd-499a-446b-9bce-a1eb301c859d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:37:06,488::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:37:06,489::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cb97593f-12b4-40cf-8440-cc3e8cf52778`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:37:06,489::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:37:06,490::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:37:06,490::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cb97593f-12b4-40cf-8440-cc3e8cf52778`::Granted request >Thread-34::DEBUG::2014-08-11 15:37:06,490::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`91beeccd-499a-446b-9bce-a1eb301c859d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:37:06,491::task::993::Storage.TaskManager.Task::(_decref) Task=`91beeccd-499a-446b-9bce-a1eb301c859d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:37:06,510::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952477696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:06,511::task::1191::Storage.TaskManager.Task::(prepare) Task=`91beeccd-499a-446b-9bce-a1eb301c859d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952477696', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:06,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`91beeccd-499a-446b-9bce-a1eb301c859d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:37:06,511::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:37:06,511::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:37:06,512::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:37:06,512::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:37:06,512::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:37:06,512::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:37:06,513::task::993::Storage.TaskManager.Task::(_decref) Task=`91beeccd-499a-446b-9bce-a1eb301c859d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:37:06,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:06,908::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131953 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:08,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:08,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132699 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:10,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:11,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137888 s, 74.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:11,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:11,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:11,406::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:11,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:37:11,458::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:37:11,458::__init__::232::IOProcess::(_processLogs) (7376) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:11,479::__init__::232::IOProcess::(_processLogs) (7376) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:11,480::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000686581 s, 663 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:11,481::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:11,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:11,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:11,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:11,488::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:11,489::__init__::232::IOProcess::(_processLogs) (7377) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:11,492::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7377) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:11,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:11,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:11,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:11,494::__init__::232::IOProcess::(_processLogs) (7378) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:11,495::__init__::232::IOProcess::(_processLogs) (7378) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:11,497::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:11,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:11,499::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:11,501::__init__::232::IOProcess::(_processLogs) (7379) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:11,503::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7379) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:11,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:11,507::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:11,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:11,510::__init__::232::IOProcess::(_processLogs) (7380) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:11,511::__init__::232::IOProcess::(_processLogs) (7380) Queuing response >Thread-115::DEBUG::2014-08-11 15:37:12,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`23f85a4b-daef-424f-bcfc-5dec32d01b96`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:12,619::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:37:12,620::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000686581', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:12,620::task::1191::Storage.TaskManager.Task::(prepare) Task=`23f85a4b-daef-424f-bcfc-5dec32d01b96`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000686581', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:12,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`23f85a4b-daef-424f-bcfc-5dec32d01b96`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:12,620::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:12,621::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:12,621::task::993::Storage.TaskManager.Task::(_decref) Task=`23f85a4b-daef-424f-bcfc-5dec32d01b96`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:37:12,641::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:37:13,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:13,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133782 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:15,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:15,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132062 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:16,496::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:16,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:37:16,528::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:37:16,558::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccffee3a-344b-4850-94c7-f3f8098f0c31`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:16,560::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:16,560::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:37:16,574::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:16,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`ccffee3a-344b-4850-94c7-f3f8098f0c31`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:16,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccffee3a-344b-4850-94c7-f3f8098f0c31`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:16,575::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:16,575::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:16,575::task::993::Storage.TaskManager.Task::(_decref) Task=`ccffee3a-344b-4850-94c7-f3f8098f0c31`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:37:16,587::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:37:16,588::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f852cc1-efdc-473b-9996-08a4897e28be`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:37:16,588::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:37:16,589::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9c847fa4-95d6-4fd6-85bd-51477f9ffbc5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:37:16,589::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:37:16,589::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:37:16,590::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9c847fa4-95d6-4fd6-85bd-51477f9ffbc5`::Granted request >Thread-34::DEBUG::2014-08-11 15:37:16,590::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3f852cc1-efdc-473b-9996-08a4897e28be`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:37:16,590::task::993::Storage.TaskManager.Task::(_decref) Task=`3f852cc1-efdc-473b-9996-08a4897e28be`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:37:16,610::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:16,611::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f852cc1-efdc-473b-9996-08a4897e28be`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:16,611::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f852cc1-efdc-473b-9996-08a4897e28be`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:37:16,611::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:37:16,611::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:37:16,612::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:37:16,612::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:37:16,612::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:37:16,612::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:37:16,613::task::993::Storage.TaskManager.Task::(_decref) Task=`3f852cc1-efdc-473b-9996-08a4897e28be`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:37:17,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:17,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133312 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:19,209::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:19,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013368 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:21,265::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:21,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133369 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:21,511::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:21,512::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:37:21,513::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:37:21,556::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:21,575::__init__::232::IOProcess::(_processLogs) (7381) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:37:21,579::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000907187 s, 502 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:21,580::__init__::232::IOProcess::(_processLogs) (7381) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:21,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:21,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:21,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:21,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:21,590::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:21,592::__init__::232::IOProcess::(_processLogs) (7382) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:21,593::__init__::232::IOProcess::(_processLogs) (7382) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:21,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:21,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:21,598::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:21,600::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:21,600::__init__::232::IOProcess::(_processLogs) (7383) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:21,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7383) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:21,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:21,602::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:21,602::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:21,603::__init__::232::IOProcess::(_processLogs) (7384) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:21,603::__init__::232::IOProcess::(_processLogs) (7384) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:21,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:21,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:21,605::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:21,605::__init__::232::IOProcess::(_processLogs) (7385) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:21,606::__init__::232::IOProcess::(_processLogs) (7385) Queuing response >Dummy-33::DEBUG::2014-08-11 15:37:23,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:23,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132343 s, 77.4 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:37:24,549::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d2bb297-a062-4ce0-bc00-79d82101e07e`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:37:24,549::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:37:24,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:24,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:24,551::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:24,551::__init__::232::IOProcess::(_processLogs) (7386) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:37:24,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7386) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:24,556::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:24,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:24,557::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:24,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7387) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:37:24,558::__init__::232::IOProcess::(_processLogs) (7387) Queuing response >Thread-122::INFO::2014-08-11 15:37:24,558::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:37:24,558::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d2bb297-a062-4ce0-bc00-79d82101e07e`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:37:24,559::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d2bb297-a062-4ce0-bc00-79d82101e07e`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:37:24,559::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:37:24,559::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:37:24,559::task::993::Storage.TaskManager.Task::(_decref) Task=`3d2bb297-a062-4ce0-bc00-79d82101e07e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:37:25,376::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:25,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0169711 s, 60.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:26,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:26,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:37:26,625::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-115::DEBUG::2014-08-11 15:37:26,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`26511596-3d9f-4ba8-9e7c-5ef6e9058f09`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:26,659::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:26,660::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:37:26,674::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:26,674::task::1191::Storage.TaskManager.Task::(prepare) Task=`26511596-3d9f-4ba8-9e7c-5ef6e9058f09`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:26,674::task::595::Storage.TaskManager.Task::(_updateState) Task=`26511596-3d9f-4ba8-9e7c-5ef6e9058f09`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:26,675::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:26,675::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:26,675::task::993::Storage.TaskManager.Task::(_decref) Task=`26511596-3d9f-4ba8-9e7c-5ef6e9058f09`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:37:26,686::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [536631e4] >Thread-34::DEBUG::2014-08-11 15:37:26,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2e8b94f-d3b7-41b9-9434-5af443f8e794`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:37:26,687::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:37:26,688::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b124d336-3d21-4c01-9cd0-3fe84e7749cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:37:26,688::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:37:26,688::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:37:26,688::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b124d336-3d21-4c01-9cd0-3fe84e7749cf`::Granted request >Thread-34::DEBUG::2014-08-11 15:37:26,689::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a2e8b94f-d3b7-41b9-9434-5af443f8e794`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:37:26,689::task::993::Storage.TaskManager.Task::(_decref) Task=`a2e8b94f-d3b7-41b9-9434-5af443f8e794`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:37:26,709::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:26,709::task::1191::Storage.TaskManager.Task::(prepare) Task=`a2e8b94f-d3b7-41b9-9434-5af443f8e794`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:26,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2e8b94f-d3b7-41b9-9434-5af443f8e794`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:37:26,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:37:26,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:37:26,710::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:37:26,711::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:37:26,711::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:37:26,711::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:37:26,711::task::993::Storage.TaskManager.Task::(_decref) Task=`a2e8b94f-d3b7-41b9-9434-5af443f8e794`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:37:27,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:27,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131945 s, 77.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:37:27,735::task::595::Storage.TaskManager.Task::(_updateState) Task=`677748b3-df06-46cf-a400-e5eea2de1f70`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:27,736::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:37:27,736::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000907187', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:27,736::task::1191::Storage.TaskManager.Task::(prepare) Task=`677748b3-df06-46cf-a400-e5eea2de1f70`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000907187', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:27,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`677748b3-df06-46cf-a400-e5eea2de1f70`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:27,737::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:27,737::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:27,737::task::993::Storage.TaskManager.Task::(_decref) Task=`677748b3-df06-46cf-a400-e5eea2de1f70`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:37:27,757::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:37:29,493::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:29,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01331 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:31,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:31,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132766 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:31,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:31,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:31,607::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:31,607::__init__::232::IOProcess::(_processLogs) (7388) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:31,608::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7388) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:31,609::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:31,674::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:31,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:31,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:37:31,679::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417816 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:31,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:31,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:31,681::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:31,681::__init__::232::IOProcess::(_processLogs) (7389) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:31,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7389) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:31,682::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:31,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:31,683::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:31,683::__init__::232::IOProcess::(_processLogs) (7390) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:31,684::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7390) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:31,685::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:31,686::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:31,686::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:31,686::__init__::232::IOProcess::(_processLogs) (7391) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:31,687::__init__::232::IOProcess::(_processLogs) (7391) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:31,687::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:31,687::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:31,688::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:31,688::__init__::232::IOProcess::(_processLogs) (7392) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:31,689::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7392) Queuing response >Dummy-33::DEBUG::2014-08-11 15:37:33,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:33,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0212859 s, 48.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:35,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:35,704::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132272 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:36,683::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:36,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:37:36,724::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:37:36,758::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc4ff57b-f623-4921-b3a5-400436c81627`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:36,759::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:36,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:37:36,774::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:36,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`cc4ff57b-f623-4921-b3a5-400436c81627`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:36,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`cc4ff57b-f623-4921-b3a5-400436c81627`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:36,775::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:36,775::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:36,775::task::993::Storage.TaskManager.Task::(_decref) Task=`cc4ff57b-f623-4921-b3a5-400436c81627`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:37:36,786::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:37:36,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`33664b82-6802-479a-9580-dfba45788732`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:37:36,787::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:37:36,787::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30db9871-7845-4b66-af69-c3add772280d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:37:36,788::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:37:36,788::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:37:36,788::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`30db9871-7845-4b66-af69-c3add772280d`::Granted request >Thread-34::DEBUG::2014-08-11 15:37:36,789::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`33664b82-6802-479a-9580-dfba45788732`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:37:36,789::task::993::Storage.TaskManager.Task::(_decref) Task=`33664b82-6802-479a-9580-dfba45788732`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:37:36,809::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:36,809::task::1191::Storage.TaskManager.Task::(prepare) Task=`33664b82-6802-479a-9580-dfba45788732`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:36,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`33664b82-6802-479a-9580-dfba45788732`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:37:36,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:37:36,810::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:37:36,810::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:37:36,811::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:37:36,811::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:37:36,811::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:37:36,811::task::993::Storage.TaskManager.Task::(_decref) Task=`33664b82-6802-479a-9580-dfba45788732`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:37:37,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:37,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134257 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:39,782::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:39,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131278 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:41,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:41,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:41,756::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:37:41,757::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:37:41,757::__init__::232::IOProcess::(_processLogs) (7393) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:41,775::__init__::232::IOProcess::(_processLogs) (7393) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:41,779::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:37:41,779::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000490958 s, 927 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:41,782::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:41,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:41,785::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:41,787::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:41,787::__init__::232::IOProcess::(_processLogs) (7394) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:41,788::__init__::232::IOProcess::(_processLogs) (7394) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:41,790::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:41,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:41,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:41,793::__init__::232::IOProcess::(_processLogs) (7395) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:41,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:41,795::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7395) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:41,797::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:41,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:41,799::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:41,799::__init__::232::IOProcess::(_processLogs) (7396) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:41,800::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7396) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:41,801::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:41,801::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:41,802::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:41,803::__init__::232::IOProcess::(_processLogs) (7397) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:41,803::__init__::232::IOProcess::(_processLogs) (7397) Queuing response >Dummy-33::DEBUG::2014-08-11 15:37:41,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:41,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132889 s, 77.1 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:37:42,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`6024841e-42d8-4224-b198-07e32ecdbbb7`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:42,854::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:37:42,854::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000490958', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:42,855::task::1191::Storage.TaskManager.Task::(prepare) Task=`6024841e-42d8-4224-b198-07e32ecdbbb7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000490958', 'lastCheck': '1.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:42,855::task::595::Storage.TaskManager.Task::(_updateState) Task=`6024841e-42d8-4224-b198-07e32ecdbbb7`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:42,855::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:42,855::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:42,855::task::993::Storage.TaskManager.Task::(_decref) Task=`6024841e-42d8-4224-b198-07e32ecdbbb7`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:37:42,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:37:43,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:43,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0294078 s, 34.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:46,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:46,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132136 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:46,805::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:37:46,829::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:37:46,858::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2c0aa3c-14da-4984-a5c6-41485ceed871`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:46,860::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:46,861::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:46,863::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:37:46,875::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:46,876::task::1191::Storage.TaskManager.Task::(prepare) Task=`c2c0aa3c-14da-4984-a5c6-41485ceed871`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:46,876::task::595::Storage.TaskManager.Task::(_updateState) Task=`c2c0aa3c-14da-4984-a5c6-41485ceed871`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:46,876::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:46,876::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:46,876::task::993::Storage.TaskManager.Task::(_decref) Task=`c2c0aa3c-14da-4984-a5c6-41485ceed871`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:37:46,888::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:37:46,888::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebad937e-a059-4b82-b182-d1413bab1bf1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:37:46,889::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:37:46,889::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0b26305c-00ed-479e-8452-573166c12da5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:37:46,890::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:37:46,890::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:37:46,890::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0b26305c-00ed-479e-8452-573166c12da5`::Granted request >Thread-34::DEBUG::2014-08-11 15:37:46,891::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ebad937e-a059-4b82-b182-d1413bab1bf1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:37:46,891::task::993::Storage.TaskManager.Task::(_decref) Task=`ebad937e-a059-4b82-b182-d1413bab1bf1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:37:46,911::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:46,911::task::1191::Storage.TaskManager.Task::(prepare) Task=`ebad937e-a059-4b82-b182-d1413bab1bf1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:46,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`ebad937e-a059-4b82-b182-d1413bab1bf1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:37:46,912::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:37:46,912::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:37:46,912::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:37:46,913::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:37:46,913::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:37:46,913::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:37:46,913::task::993::Storage.TaskManager.Task::(_decref) Task=`ebad937e-a059-4b82-b182-d1413bab1bf1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:37:48,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:48,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131956 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:50,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:50,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133148 s, 76.9 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:37:51,803::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:37:51,805::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,805::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,855::__init__::232::IOProcess::(_processLogs) (7398) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:37:51,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7398) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:51,866::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:37:51,867::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,867::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,868::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,868::__init__::232::IOProcess::(_processLogs) (7399) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:37:51,868::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7399) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:51,870::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:37:51,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:51,872::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:51,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,874::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,876::__init__::232::IOProcess::(_processLogs) (7400) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:37:51,878::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7400) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:51,878::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:51,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:51,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,884::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:51,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:37:51,886::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,887::__init__::232::IOProcess::(_processLogs) (7401) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:51,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7401) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:51,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,892::__init__::232::IOProcess::(_processLogs) (7402) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:37:51,892::__init__::232::IOProcess::(_processLogs) (7402) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:51,892::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:37:51,893::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:37:51,894::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:37:51,894::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,895::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,895::__init__::232::IOProcess::(_processLogs) (7403) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:51,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7403) Queuing response >Thread-16::DEBUG::2014-08-11 15:37:51,897::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:37:51,918::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00173372 s, 262 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:37:51,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,920::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,921::__init__::232::IOProcess::(_processLogs) (7404) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:37:51,921::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7404) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:51,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,924::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,924::__init__::232::IOProcess::(_processLogs) (7405) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:51,924::__init__::232::IOProcess::(_processLogs) (7405) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:51,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,926::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,926::__init__::232::IOProcess::(_processLogs) (7406) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:51,927::__init__::232::IOProcess::(_processLogs) (7406) Queuing response >Thread-14::DEBUG::2014-08-11 15:37:51,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:37:51,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:37:51,928::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:37:51,929::__init__::232::IOProcess::(_processLogs) (7407) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:37:51,929::__init__::232::IOProcess::(_processLogs) (7407) Queuing response >Dummy-33::DEBUG::2014-08-11 15:37:52,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:52,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131461 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:54,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:54,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132193 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:37:56,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:56,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132518 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:56,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:37:56,934::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:37:56,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`5eb7c87d-57a1-4378-bc77-a26024bf24f2`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:56,959::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:56,961::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:37:56,962::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:37:56,975::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:56,976::task::1191::Storage.TaskManager.Task::(prepare) Task=`5eb7c87d-57a1-4378-bc77-a26024bf24f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:37:56,976::task::595::Storage.TaskManager.Task::(_updateState) Task=`5eb7c87d-57a1-4378-bc77-a26024bf24f2`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:56,976::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:56,976::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:56,977::task::993::Storage.TaskManager.Task::(_decref) Task=`5eb7c87d-57a1-4378-bc77-a26024bf24f2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:37:56,987::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:37:56,988::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a3a0dba-1138-4031-b7a0-2a5ec9cc3d07`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:37:56,988::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:37:56,991::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d28780d-c7c4-4a7d-8420-e660e9d762c2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:37:56,991::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:37:56,992::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:37:56,992::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d28780d-c7c4-4a7d-8420-e660e9d762c2`::Granted request >Thread-34::DEBUG::2014-08-11 15:37:56,992::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1a3a0dba-1138-4031-b7a0-2a5ec9cc3d07`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:37:56,993::task::993::Storage.TaskManager.Task::(_decref) Task=`1a3a0dba-1138-4031-b7a0-2a5ec9cc3d07`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:37:57,013::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:57,013::task::1191::Storage.TaskManager.Task::(prepare) Task=`1a3a0dba-1138-4031-b7a0-2a5ec9cc3d07`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:37:57,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`1a3a0dba-1138-4031-b7a0-2a5ec9cc3d07`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:37:57,014::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:37:57,014::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:37:57,014::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:37:57,015::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:37:57,015::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:37:57,015::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:37:57,015::task::993::Storage.TaskManager.Task::(_decref) Task=`1a3a0dba-1138-4031-b7a0-2a5ec9cc3d07`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:37:58,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`552288b1-e71a-4da9-b98c-e7b978e203c3`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:37:58,004::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:37:58,004::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00173372', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:58,005::task::1191::Storage.TaskManager.Task::(prepare) Task=`552288b1-e71a-4da9-b98c-e7b978e203c3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00173372', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:37:58,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`552288b1-e71a-4da9-b98c-e7b978e203c3`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:37:58,005::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:37:58,005::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:37:58,005::task::993::Storage.TaskManager.Task::(_decref) Task=`552288b1-e71a-4da9-b98c-e7b978e203c3`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:37:58,028::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:37:58,467::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:37:58,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132417 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:00,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:00,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132994 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:01,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:01,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:01,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:01,957::__init__::232::IOProcess::(_processLogs) (7408) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:01,958::__init__::232::IOProcess::(_processLogs) (7408) Queuing response >Thread-16::DEBUG::2014-08-11 15:38:01,965::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:01,987::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:38:01,987::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000725109 s, 627 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:01,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:01,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:01,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:01,997::__init__::232::IOProcess::(_processLogs) (7409) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:01,999::__init__::232::IOProcess::(_processLogs) (7409) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:01,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:38:02,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:02,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:02,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:02,008::__init__::232::IOProcess::(_processLogs) (7410) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:02,009::__init__::232::IOProcess::(_processLogs) (7410) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:02,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:02,013::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:02,014::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:02,014::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:38:02,016::__init__::232::IOProcess::(_processLogs) (7411) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:02,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7411) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:02,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:02,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:02,024::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:02,026::__init__::232::IOProcess::(_processLogs) (7412) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:02,027::__init__::232::IOProcess::(_processLogs) (7412) Queuing response >Dummy-33::DEBUG::2014-08-11 15:38:02,578::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:02,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283244 s, 36.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:04,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:04,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133365 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:06,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:06,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132462 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:07,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:07,025::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:07,026::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:38:07,032::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:38:07,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce729d3d-7abd-4218-be3c-34e942ed9006`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:07,033::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:38:07,045::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:07,046::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce729d3d-7abd-4218-be3c-34e942ed9006`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:07,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce729d3d-7abd-4218-be3c-34e942ed9006`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:07,046::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:07,047::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:07,047::task::993::Storage.TaskManager.Task::(_decref) Task=`ce729d3d-7abd-4218-be3c-34e942ed9006`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:38:07,057::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:38:07,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`de0e3aae-5edd-416c-a58e-b1f9eb5d9489`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:38:07,058::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:38:07,058::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`71e492f2-0ddb-4319-93e7-d9f48c2bc93d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:38:07,059::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:38:07,059::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:38:07,059::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`71e492f2-0ddb-4319-93e7-d9f48c2bc93d`::Granted request >Thread-34::DEBUG::2014-08-11 15:38:07,060::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`de0e3aae-5edd-416c-a58e-b1f9eb5d9489`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:38:07,060::task::993::Storage.TaskManager.Task::(_decref) Task=`de0e3aae-5edd-416c-a58e-b1f9eb5d9489`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:38:07,080::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:07,080::task::1191::Storage.TaskManager.Task::(prepare) Task=`de0e3aae-5edd-416c-a58e-b1f9eb5d9489`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:07,081::task::595::Storage.TaskManager.Task::(_updateState) Task=`de0e3aae-5edd-416c-a58e-b1f9eb5d9489`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:38:07,081::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:38:07,081::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:38:07,081::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:38:07,082::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:38:07,082::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:38:07,082::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:38:07,082::task::993::Storage.TaskManager.Task::(_decref) Task=`de0e3aae-5edd-416c-a58e-b1f9eb5d9489`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:38:08,958::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:08,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131691 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:11,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:11,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133015 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:12,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:12,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:12,033::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:38:12,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:38:12,059::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:38:12,060::__init__::232::IOProcess::(_processLogs) (7413) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:12,080::__init__::232::IOProcess::(_processLogs) (7413) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:12,083::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:12,084::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:38:12,085::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000850627 s, 535 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:12,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:12,089::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:12,089::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:12,089::__init__::232::IOProcess::(_processLogs) (7414) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:12,090::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7414) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:12,091::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:12,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:12,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:12,092::__init__::232::IOProcess::(_processLogs) (7415) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:12,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7415) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:12,093::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:12,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:12,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:12,094::__init__::232::IOProcess::(_processLogs) (7416) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:12,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7416) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:12,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:12,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:12,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:12,096::__init__::232::IOProcess::(_processLogs) (7417) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:12,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7417) Queuing response >Dummy-33::DEBUG::2014-08-11 15:38:13,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:13,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.017821 s, 57.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:38:13,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2b0e744-fd7b-4d3b-81da-40c1d24f86f7`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:13,125::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:38:13,126::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000850627', 'lastCheck': '1.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:13,126::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2b0e744-fd7b-4d3b-81da-40c1d24f86f7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000850627', 'lastCheck': '1.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:13,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2b0e744-fd7b-4d3b-81da-40c1d24f86f7`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:13,127::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:13,128::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:13,129::task::993::Storage.TaskManager.Task::(_decref) Task=`f2b0e744-fd7b-4d3b-81da-40c1d24f86f7`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:38:13,170::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:38:15,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:15,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0157305 s, 65.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:17,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:17,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:38:17,098::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:38:17,159::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf52ed4c-ce99-45a0-9f01-e9a967117edd`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:17,161::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:38:17,161::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:38:17,176::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:17,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf52ed4c-ce99-45a0-9f01-e9a967117edd`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:17,176::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf52ed4c-ce99-45a0-9f01-e9a967117edd`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:17,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:17,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:17,177::task::993::Storage.TaskManager.Task::(_decref) Task=`bf52ed4c-ce99-45a0-9f01-e9a967117edd`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:38:17,187::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:38:17,187::task::595::Storage.TaskManager.Task::(_updateState) Task=`6225a1a4-010a-4721-b9fc-02ff34d3522f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:38:17,188::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:38:17,189::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9e9eedb-5240-465f-8fa6-7fc6b62aae8f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:38:17,189::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:38:17,189::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:38:17,189::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e9e9eedb-5240-465f-8fa6-7fc6b62aae8f`::Granted request >Thread-34::DEBUG::2014-08-11 15:38:17,190::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6225a1a4-010a-4721-b9fc-02ff34d3522f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:38:17,190::task::993::Storage.TaskManager.Task::(_decref) Task=`6225a1a4-010a-4721-b9fc-02ff34d3522f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:38:17,210::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:17,210::task::1191::Storage.TaskManager.Task::(prepare) Task=`6225a1a4-010a-4721-b9fc-02ff34d3522f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952412160', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:17,211::task::595::Storage.TaskManager.Task::(_updateState) Task=`6225a1a4-010a-4721-b9fc-02ff34d3522f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:38:17,211::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:38:17,211::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:38:17,211::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:38:17,212::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:38:17,212::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:38:17,212::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:38:17,212::task::993::Storage.TaskManager.Task::(_decref) Task=`6225a1a4-010a-4721-b9fc-02ff34d3522f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:38:17,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:17,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133016 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:19,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:19,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134024 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:21,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:21,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133279 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:22,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:22,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:22,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:38:22,158::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:38:22,159::__init__::232::IOProcess::(_processLogs) (7418) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:22,178::__init__::232::IOProcess::(_processLogs) (7418) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:22,181::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:38:22,182::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00054787 s, 830 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:22,185::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:22,187::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:38:22,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:22,189::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:22,189::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:22,189::__init__::232::IOProcess::(_processLogs) (7419) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:22,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7419) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:22,191::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:22,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:22,193::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:22,193::__init__::232::IOProcess::(_processLogs) (7420) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:22,193::__init__::232::IOProcess::(_processLogs) (7420) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:22,193::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:22,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:22,195::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:22,195::__init__::232::IOProcess::(_processLogs) (7421) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:22,195::__init__::232::IOProcess::(_processLogs) (7421) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:22,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:22,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:22,197::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:22,198::__init__::232::IOProcess::(_processLogs) (7422) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:22,198::__init__::232::IOProcess::(_processLogs) (7422) Queuing response >Dummy-33::DEBUG::2014-08-11 15:38:23,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:23,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132092 s, 77.5 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:38:24,663::task::595::Storage.TaskManager.Task::(_updateState) Task=`5466221f-b5c6-4eb0-9166-437c4913458d`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:38:24,664::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:38:24,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:38:24,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:24,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:24,666::__init__::232::IOProcess::(_processLogs) (7423) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:38:24,667::__init__::232::IOProcess::(_processLogs) (7424) Got request for method 'stat' >Thread-122::INFO::2014-08-11 15:38:24,668::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:38:24,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`5466221f-b5c6-4eb0-9166-437c4913458d`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:38:24,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`5466221f-b5c6-4eb0-9166-437c4913458d`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:38:24,669::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:38:24,669::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:38:24,669::task::993::Storage.TaskManager.Task::(_decref) Task=`5466221f-b5c6-4eb0-9166-437c4913458d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:38:25,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:25,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132757 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:27,189::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:27,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:38:27,225::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:38:27,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2412e46-7366-4692-b7d5-5af98282f195`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:27,261::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:38:27,261::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:38:27,276::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:27,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2412e46-7366-4692-b7d5-5af98282f195`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:27,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2412e46-7366-4692-b7d5-5af98282f195`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:27,276::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:27,277::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:27,277::task::993::Storage.TaskManager.Task::(_decref) Task=`b2412e46-7366-4692-b7d5-5af98282f195`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:38:27,287::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:38:27,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`99a380c3-5496-4576-98e5-7af2e6eb791f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:38:27,288::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:38:27,289::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6ecae09c-5272-405a-8c4e-d726e0698873`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:38:27,289::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:38:27,289::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:38:27,289::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6ecae09c-5272-405a-8c4e-d726e0698873`::Granted request >Thread-34::DEBUG::2014-08-11 15:38:27,290::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`99a380c3-5496-4576-98e5-7af2e6eb791f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:38:27,290::task::993::Storage.TaskManager.Task::(_decref) Task=`99a380c3-5496-4576-98e5-7af2e6eb791f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:38:27,310::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:27,310::task::1191::Storage.TaskManager.Task::(prepare) Task=`99a380c3-5496-4576-98e5-7af2e6eb791f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:27,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`99a380c3-5496-4576-98e5-7af2e6eb791f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:38:27,311::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:38:27,311::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:38:27,311::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:38:27,312::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:38:27,312::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:38:27,312::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:38:27,313::task::993::Storage.TaskManager.Task::(_decref) Task=`99a380c3-5496-4576-98e5-7af2e6eb791f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:38:27,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:27,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132005 s, 77.6 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:38:28,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`b65d657c-5a2d-43b6-8b77-d66cf0671583`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:28,298::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:38:28,299::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00054787', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:28,299::task::1191::Storage.TaskManager.Task::(prepare) Task=`b65d657c-5a2d-43b6-8b77-d66cf0671583`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00054787', 'lastCheck': '6.1', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:28,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`b65d657c-5a2d-43b6-8b77-d66cf0671583`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:28,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:28,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:28,300::task::993::Storage.TaskManager.Task::(_decref) Task=`b65d657c-5a2d-43b6-8b77-d66cf0671583`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:38:28,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:38:29,550::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:29,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013362 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:31,606::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:31,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133641 s, 76.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:32,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:32,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:32,257::__init__::232::IOProcess::(_processLogs) (7425) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:38:32,260::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:32,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:38:32,283::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000459813 s, 990 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:32,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:32,288::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:38:32,290::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:32,291::__init__::232::IOProcess::(_processLogs) (7426) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:32,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:38:32,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:32,298::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:32,300::__init__::232::IOProcess::(_processLogs) (7427) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:32,304::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:32,306::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:32,307::__init__::232::IOProcess::(_processLogs) (7428) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:32,309::__init__::232::IOProcess::(_processLogs) (7428) Queuing response >Thread-14::DEBUG::2014-08-11 15:38:32,311::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:38:32,313::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:38:32,315::__init__::232::IOProcess::(_processLogs) (7429) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:38:33,664::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:33,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132735 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:35,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:35,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131676 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:37,303::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:37,304::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:38:37,328::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:38:37,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`368345de-4d68-46e0-a59d-145907e268e9`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:37,361::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:37,361::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:38:37,376::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:37,376::task::1191::Storage.TaskManager.Task::(prepare) Task=`368345de-4d68-46e0-a59d-145907e268e9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:37,376::task::595::Storage.TaskManager.Task::(_updateState) Task=`368345de-4d68-46e0-a59d-145907e268e9`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:37,377::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:37,377::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:37,377::task::993::Storage.TaskManager.Task::(_decref) Task=`368345de-4d68-46e0-a59d-145907e268e9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:38:37,389::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:38:37,390::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0da9223-fe5e-494c-9487-7cf5ca9580f1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:38:37,390::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:38:37,391::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a896e76-9ab2-4412-83a0-479821a7bc1a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:38:37,391::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:38:37,391::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:38:37,392::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1a896e76-9ab2-4412-83a0-479821a7bc1a`::Granted request >Thread-34::DEBUG::2014-08-11 15:38:37,392::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e0da9223-fe5e-494c-9487-7cf5ca9580f1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:38:37,392::task::993::Storage.TaskManager.Task::(_decref) Task=`e0da9223-fe5e-494c-9487-7cf5ca9580f1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:38:37,418::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:37,418::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0da9223-fe5e-494c-9487-7cf5ca9580f1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:37,419::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0da9223-fe5e-494c-9487-7cf5ca9580f1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:38:37,419::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:38:37,419::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:38:37,419::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:38:37,420::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:38:37,420::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:38:37,420::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:38:37,420::task::993::Storage.TaskManager.Task::(_decref) Task=`e0da9223-fe5e-494c-9487-7cf5ca9580f1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:38:37,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:37,808::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133142 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:39,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:39,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132167 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:41,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:41,917::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131526 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:42,320::__init__::232::IOProcess::(_processLogs) (7430) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:38:42,321::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:42,375::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:42,376::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:42,377::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:38:42,379::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420235 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:42,380::__init__::232::IOProcess::(_processLogs) (7431) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:42,382::__init__::232::IOProcess::(_processLogs) (7432) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:42,383::__init__::232::IOProcess::(_processLogs) (7433) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:42,384::__init__::232::IOProcess::(_processLogs) (7434) Got request for method 'access' >Thread-115::DEBUG::2014-08-11 15:38:43,432::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb608417-0436-44e2-951f-9a7adc07a6b7`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:43,432::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:38:43,432::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420235', 'lastCheck': '1.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:43,433::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb608417-0436-44e2-951f-9a7adc07a6b7`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420235', 'lastCheck': '1.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:43,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb608417-0436-44e2-951f-9a7adc07a6b7`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:43,433::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:43,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:43,434::task::993::Storage.TaskManager.Task::(_decref) Task=`eb608417-0436-44e2-951f-9a7adc07a6b7`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:38:43,454::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:38:43,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:43,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133045 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:45,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:46,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132612 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:47,384::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:47,385::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:47,386::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::DEBUG::2014-08-11 15:38:47,433::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:38:47,434::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4699a68-480a-4ea0-b234-680b3227cb95`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:47,434::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:38:47,447::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:47,447::task::1191::Storage.TaskManager.Task::(prepare) Task=`e4699a68-480a-4ea0-b234-680b3227cb95`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:47,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`e4699a68-480a-4ea0-b234-680b3227cb95`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:47,448::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:47,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:47,448::task::993::Storage.TaskManager.Task::(_decref) Task=`e4699a68-480a-4ea0-b234-680b3227cb95`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:38:47,458::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:38:47,459::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed4b741a-45b3-4dff-8779-fc6b5a9d882b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:38:47,459::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:38:47,460::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0ae0ee06-a2fe-4d02-bb36-978e4303c465`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:38:47,460::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:38:47,461::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:38:47,461::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0ae0ee06-a2fe-4d02-bb36-978e4303c465`::Granted request >Thread-34::DEBUG::2014-08-11 15:38:47,461::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ed4b741a-45b3-4dff-8779-fc6b5a9d882b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:38:47,462::task::993::Storage.TaskManager.Task::(_decref) Task=`ed4b741a-45b3-4dff-8779-fc6b5a9d882b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:38:47,481::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:47,482::task::1191::Storage.TaskManager.Task::(prepare) Task=`ed4b741a-45b3-4dff-8779-fc6b5a9d882b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:47,482::task::595::Storage.TaskManager.Task::(_updateState) Task=`ed4b741a-45b3-4dff-8779-fc6b5a9d882b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:38:47,482::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:38:47,482::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:38:47,483::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:38:47,483::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:38:47,483::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:38:47,483::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:38:47,484::task::993::Storage.TaskManager.Task::(_decref) Task=`ed4b741a-45b3-4dff-8779-fc6b5a9d882b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:38:48,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:48,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133603 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:50,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:50,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133374 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:52,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:52,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131665 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:52,386::__init__::232::IOProcess::(_processLogs) (7435) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:38:52,387::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:52,392::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:52,578::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:52,579::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:38:52,581::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0035229 s, 129 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:38:52,582::__init__::232::IOProcess::(_processLogs) (7436) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:38:52,583::__init__::232::IOProcess::(_processLogs) (7437) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:52,584::__init__::232::IOProcess::(_processLogs) (7438) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:38:52,585::__init__::232::IOProcess::(_processLogs) (7439) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:38:54,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:54,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131893 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:38:56,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:56,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132946 s, 77.0 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:38:57,496::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:38:57,497::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ef8d7f6-9394-4bb0-be1e-b815cbba454a`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:57,558::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-115::INFO::2014-08-11 15:38:57,571::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:57,572::task::1191::Storage.TaskManager.Task::(prepare) Task=`8ef8d7f6-9394-4bb0-be1e-b815cbba454a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:38:57,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`8ef8d7f6-9394-4bb0-be1e-b815cbba454a`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:57,572::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:57,572::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:57,572::task::993::Storage.TaskManager.Task::(_decref) Task=`8ef8d7f6-9394-4bb0-be1e-b815cbba454a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:38:57,583::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:38:57,583::task::595::Storage.TaskManager.Task::(_updateState) Task=`43fde04f-caf3-4812-ad53-b637fc864aa2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:38:57,584::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:38:57,586::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`307dfa69-ed40-47cc-a5a3-7f9f52ec71ca`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:57,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:38:57,587::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:38:57,589::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:38:57,591::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`307dfa69-ed40-47cc-a5a3-7f9f52ec71ca`::Granted request >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:57,593::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:38:57,594::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`43fde04f-caf3-4812-ad53-b637fc864aa2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:38:57,595::task::993::Storage.TaskManager.Task::(_decref) Task=`43fde04f-caf3-4812-ad53-b637fc864aa2`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:38:57,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:38:57,621::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:57,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`43fde04f-caf3-4812-ad53-b637fc864aa2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952346624', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:38:57,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`43fde04f-caf3-4812-ad53-b637fc864aa2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:38:57,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:38:57,622::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:38:57,622::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:38:57,623::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:38:57,623::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:38:57,623::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:38:57,623::task::993::Storage.TaskManager.Task::(_decref) Task=`43fde04f-caf3-4812-ad53-b637fc864aa2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:38:58,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:38:58,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133457 s, 76.7 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:38:58,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`76e2af74-6fdb-4917-bd1a-4a8403397b77`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:38:58,546::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-115::INFO::2014-08-11 15:38:58,546::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0035229', 'lastCheck': '6.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:58,546::task::1191::Storage.TaskManager.Task::(prepare) Task=`76e2af74-6fdb-4917-bd1a-4a8403397b77`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0035229', 'lastCheck': '6.0', 'valid': True}} >Thread-115::DEBUG::2014-08-11 15:38:58,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`76e2af74-6fdb-4917-bd1a-4a8403397b77`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:38:58,547::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:38:58,547::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:38:58,547::task::993::Storage.TaskManager.Task::(_decref) Task=`76e2af74-6fdb-4917-bd1a-4a8403397b77`::ref 0 aborting False >Thread-115::DEBUG::2014-08-11 15:38:58,571::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:39:00,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:00,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133091 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:02,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:02,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268391 s, 38.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:02,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:02,590::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:02,591::__init__::232::IOProcess::(_processLogs) (7440) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:39:02,594::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:02,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:39:02,621::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00102472 s, 444 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:02,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:02,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:02,631::__init__::232::IOProcess::(_processLogs) (7441) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:02,636::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:39:02,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:02,643::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:02,645::__init__::232::IOProcess::(_processLogs) (7442) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:02,648::__init__::232::IOProcess::(_processLogs) (7442) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:02,653::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:39:02,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:02,657::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:02,660::__init__::232::IOProcess::(_processLogs) (7443) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:02,663::__init__::232::IOProcess::(_processLogs) (7443) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:02,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:02,668::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:02,670::__init__::232::IOProcess::(_processLogs) (7444) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:02,671::__init__::232::IOProcess::(_processLogs) (7444) Queuing response >Dummy-33::DEBUG::2014-08-11 15:39:04,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:04,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134856 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:06,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:06,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132061 s, 77.5 MB/s\n'; <rc> = 0 >Thread-115::DEBUG::2014-08-11 15:39:07,637::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-115::DEBUG::2014-08-11 15:39:07,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`00c30aaf-17fb-4f43-8833-bd409d88568d`::moving from state init -> state preparing >Thread-115::INFO::2014-08-11 15:39:07,658::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:07,670::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:07,672::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:07,673::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-115::INFO::2014-08-11 15:39:07,674::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:39:07,675::task::1191::Storage.TaskManager.Task::(prepare) Task=`00c30aaf-17fb-4f43-8833-bd409d88568d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-115::DEBUG::2014-08-11 15:39:07,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`00c30aaf-17fb-4f43-8833-bd409d88568d`::moving from state preparing -> state finished >Thread-115::DEBUG::2014-08-11 15:39:07,675::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-115::DEBUG::2014-08-11 15:39:07,675::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-115::DEBUG::2014-08-11 15:39:07,675::task::993::Storage.TaskManager.Task::(_decref) Task=`00c30aaf-17fb-4f43-8833-bd409d88568d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:39:07,688::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:39:07,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`154428c3-8d89-43f2-9b3e-e5f7ece8e327`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:39:07,689::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:39:07,689::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fc0465f8-068c-4414-90eb-6a57c605d12a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:39:07,690::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:39:07,690::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:39:07,690::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`fc0465f8-068c-4414-90eb-6a57c605d12a`::Granted request >Thread-34::DEBUG::2014-08-11 15:39:07,691::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`154428c3-8d89-43f2-9b3e-e5f7ece8e327`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:39:07,691::task::993::Storage.TaskManager.Task::(_decref) Task=`154428c3-8d89-43f2-9b3e-e5f7ece8e327`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:39:07,711::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:07,711::task::1191::Storage.TaskManager.Task::(prepare) Task=`154428c3-8d89-43f2-9b3e-e5f7ece8e327`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:07,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`154428c3-8d89-43f2-9b3e-e5f7ece8e327`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:39:07,712::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:39:07,712::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:39:07,712::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:39:07,712::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:39:07,713::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:39:07,713::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:39:07,713::task::993::Storage.TaskManager.Task::(_decref) Task=`154428c3-8d89-43f2-9b3e-e5f7ece8e327`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 15:39:07,845::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:34849 >Detector thread::DEBUG::2014-08-11 15:39:07,848::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:34849 >Detector thread::DEBUG::2014-08-11 15:39:07,848::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:34849 >Detector thread::DEBUG::2014-08-11 15:39:07,848::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 34849) >Dummy-33::DEBUG::2014-08-11 15:39:08,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:08,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133364 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:10,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:10,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133079 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:12,668::__init__::232::IOProcess::(_processLogs) (7445) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:39:12,669::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:12,679::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:12,780::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:12,781::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:39:12,782::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00305487 s, 149 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:12,784::__init__::232::IOProcess::(_processLogs) (7446) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:39:12,785::__init__::232::IOProcess::(_processLogs) (7447) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:12,786::__init__::232::IOProcess::(_processLogs) (7448) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:12,787::__init__::232::IOProcess::(_processLogs) (7449) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:39:13,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:13,052::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131896 s, 77.6 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:39:13,883::task::595::Storage.TaskManager.Task::(_updateState) Task=`adc22dd9-fb20-42fa-857b-0a367139944a`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:13,883::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:39:13,884::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00305487', 'lastCheck': '1.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:13,884::task::1191::Storage.TaskManager.Task::(prepare) Task=`adc22dd9-fb20-42fa-857b-0a367139944a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00305487', 'lastCheck': '1.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:13,884::task::595::Storage.TaskManager.Task::(_updateState) Task=`adc22dd9-fb20-42fa-857b-0a367139944a`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:13,884::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:13,885::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:13,885::task::993::Storage.TaskManager.Task::(_decref) Task=`adc22dd9-fb20-42fa-857b-0a367139944a`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:39:13,912::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:39:15,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:15,108::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132742 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:17,131::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:17,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132865 s, 77.1 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:39:17,740::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:39:17,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`c14258a5-4311-46dc-b134-a68795c39d2d`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:17,758::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:39:17,771::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:17,772::task::1191::Storage.TaskManager.Task::(prepare) Task=`c14258a5-4311-46dc-b134-a68795c39d2d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:17,772::task::595::Storage.TaskManager.Task::(_updateState) Task=`c14258a5-4311-46dc-b134-a68795c39d2d`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:17,772::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:17,772::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:17,773::task::993::Storage.TaskManager.Task::(_decref) Task=`c14258a5-4311-46dc-b134-a68795c39d2d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:39:17,783::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:39:17,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`4057182c-82b7-4128-aabf-7aa0f978d3c7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:39:17,784::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:39:17,785::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0ecca01-704e-4aa2-ab53-138470c73d3c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:39:17,785::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:39:17,785::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:39:17,785::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d0ecca01-704e-4aa2-ab53-138470c73d3c`::Granted request >Thread-34::DEBUG::2014-08-11 15:39:17,786::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4057182c-82b7-4128-aabf-7aa0f978d3c7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:39:17,787::task::993::Storage.TaskManager.Task::(_decref) Task=`4057182c-82b7-4128-aabf-7aa0f978d3c7`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:17,789::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:17,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:17,800::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:39:17,821::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:17,821::task::1191::Storage.TaskManager.Task::(prepare) Task=`4057182c-82b7-4128-aabf-7aa0f978d3c7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:17,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`4057182c-82b7-4128-aabf-7aa0f978d3c7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:39:17,822::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:39:17,822::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:39:17,822::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:39:17,823::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:39:17,823::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:39:17,823::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:39:17,823::task::993::Storage.TaskManager.Task::(_decref) Task=`4057182c-82b7-4128-aabf-7aa0f978d3c7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:39:19,186::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:19,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133423 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:21,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:21,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132243 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:22,789::__init__::232::IOProcess::(_processLogs) (7450) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:39:22,790::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:22,807::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:39:22,879::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000566025 s, 804 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:22,881::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:22,882::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:22,883::__init__::232::IOProcess::(_processLogs) (7451) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:22,883::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:39:22,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:22,887::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:22,889::__init__::232::IOProcess::(_processLogs) (7452) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:22,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:39:22,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:22,895::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:22,895::__init__::232::IOProcess::(_processLogs) (7453) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:22,895::__init__::232::IOProcess::(_processLogs) (7453) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:22,896::__init__::232::IOProcess::(_processLogs) (7454) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:39:23,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:23,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013269 s, 77.2 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:39:24,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fed0aff-794a-4abc-925f-8e861eaa9847`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:39:24,782::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:39:24,783::__init__::232::IOProcess::(_processLogs) (7455) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:39:24,785::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:24,785::__init__::232::IOProcess::(_processLogs) (7456) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:39:24,786::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7456) Queuing response >Thread-122::INFO::2014-08-11 15:39:24,786::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:39:24,787::task::1191::Storage.TaskManager.Task::(prepare) Task=`4fed0aff-794a-4abc-925f-8e861eaa9847`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:39:24,787::task::595::Storage.TaskManager.Task::(_updateState) Task=`4fed0aff-794a-4abc-925f-8e861eaa9847`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:39:24,787::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:39:24,787::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:39:24,787::task::993::Storage.TaskManager.Task::(_decref) Task=`4fed0aff-794a-4abc-925f-8e861eaa9847`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:39:25,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:25,386::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131871 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:27,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:27,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134043 s, 76.4 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:39:27,836::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:39:27,837::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a0e2487-3f0b-4c6a-bde0-2c6c7c1e5d37`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:27,837::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:39:27,850::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:27,851::task::1191::Storage.TaskManager.Task::(prepare) Task=`4a0e2487-3f0b-4c6a-bde0-2c6c7c1e5d37`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:27,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`4a0e2487-3f0b-4c6a-bde0-2c6c7c1e5d37`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:27,851::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:27,851::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:27,852::task::993::Storage.TaskManager.Task::(_decref) Task=`4a0e2487-3f0b-4c6a-bde0-2c6c7c1e5d37`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:39:27,862::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:39:27,862::task::595::Storage.TaskManager.Task::(_updateState) Task=`8aa8b91b-1b68-4bf4-8bf4-7f296c452466`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:39:27,863::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:39:27,863::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9261ece8-7091-4988-855c-9412a151ae77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:39:27,864::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:39:27,864::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:39:27,864::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9261ece8-7091-4988-855c-9412a151ae77`::Granted request >Thread-34::DEBUG::2014-08-11 15:39:27,865::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8aa8b91b-1b68-4bf4-8bf4-7f296c452466`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:39:27,865::task::993::Storage.TaskManager.Task::(_decref) Task=`8aa8b91b-1b68-4bf4-8bf4-7f296c452466`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:39:27,885::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:27,885::task::1191::Storage.TaskManager.Task::(prepare) Task=`8aa8b91b-1b68-4bf4-8bf4-7f296c452466`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:27,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`8aa8b91b-1b68-4bf4-8bf4-7f296c452466`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:39:27,886::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:39:27,886::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:39:27,886::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:39:27,887::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:39:27,887::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:39:27,887::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:39:27,887::task::993::Storage.TaskManager.Task::(_decref) Task=`8aa8b91b-1b68-4bf4-8bf4-7f296c452466`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:27,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:27,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:27,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:39:29,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8d3e2b2-f369-4179-904d-f54a73d4ad02`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:29,003::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:39:29,003::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000566025', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:29,004::task::1191::Storage.TaskManager.Task::(prepare) Task=`d8d3e2b2-f369-4179-904d-f54a73d4ad02`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000566025', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:29,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`d8d3e2b2-f369-4179-904d-f54a73d4ad02`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:29,004::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:29,004::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:29,004::task::993::Storage.TaskManager.Task::(_decref) Task=`d8d3e2b2-f369-4179-904d-f54a73d4ad02`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:39:29,027::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:39:29,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:29,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133201 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:31,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:31,555::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131664 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:32,898::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:39:32,900::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:32,909::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:39:32,960::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:39:32,962::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:39:32,962::__init__::232::IOProcess::(_processLogs) (7457) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:32,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:39:32,984::__init__::232::IOProcess::(_processLogs) (7457) Queuing response >Thread-16::DEBUG::2014-08-11 15:39:32,986::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00103626 s, 439 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:32,989::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:32,990::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:39:32,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:32,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:32,994::__init__::232::IOProcess::(_processLogs) (7458) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:39:32,995::__init__::232::IOProcess::(_processLogs) DEBUG|(7458) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:32,996::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:39:32,996::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:32,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:32,997::__init__::232::IOProcess::(_processLogs) DEBUG|(7459) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:32,997::__init__::232::IOProcess::(_processLogs) (7459) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:32,998::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:39:32,998::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:32,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:32,998::__init__::232::IOProcess::(_processLogs) DEBUG|(7460) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:32,999::__init__::232::IOProcess::(_processLogs) (7460) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:32,999::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:39:33,000::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:33,000::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:33,000::__init__::232::IOProcess::(_processLogs) DEBUG|(7461) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:33,001::__init__::232::IOProcess::(_processLogs) (7461) Queuing response >Dummy-33::DEBUG::2014-08-11 15:39:33,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:33,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134863 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:35,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:35,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134123 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:37,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:37,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131377 s, 77.9 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:39:37,901::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:39:37,902::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fb1c2c9-889b-4c34-9a64-63b80feeba6c`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:37,903::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:39:37,916::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:37,916::task::1191::Storage.TaskManager.Task::(prepare) Task=`3fb1c2c9-889b-4c34-9a64-63b80feeba6c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:37,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`3fb1c2c9-889b-4c34-9a64-63b80feeba6c`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:37,917::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:37,917::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:37,917::task::993::Storage.TaskManager.Task::(_decref) Task=`3fb1c2c9-889b-4c34-9a64-63b80feeba6c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:39:37,928::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:39:37,929::task::595::Storage.TaskManager.Task::(_updateState) Task=`685dfe8c-60ae-41e0-9cdd-c242799a1411`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:39:37,929::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:39:37,930::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8c053af-e70a-4626-b357-ab42cf172aff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:39:37,930::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:39:37,931::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:39:37,931::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e8c053af-e70a-4626-b357-ab42cf172aff`::Granted request >Thread-34::DEBUG::2014-08-11 15:39:37,932::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`685dfe8c-60ae-41e0-9cdd-c242799a1411`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:39:37,932::task::993::Storage.TaskManager.Task::(_decref) Task=`685dfe8c-60ae-41e0-9cdd-c242799a1411`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:39:37,952::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:37,952::task::1191::Storage.TaskManager.Task::(prepare) Task=`685dfe8c-60ae-41e0-9cdd-c242799a1411`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:37,952::task::595::Storage.TaskManager.Task::(_updateState) Task=`685dfe8c-60ae-41e0-9cdd-c242799a1411`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:39:37,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:39:37,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:39:37,953::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:39:37,954::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:39:37,954::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:39:37,954::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:39:37,954::task::993::Storage.TaskManager.Task::(_decref) Task=`685dfe8c-60ae-41e0-9cdd-c242799a1411`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:38,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:38,001::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:38,002::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:39:39,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:39,777::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132209 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:41,800::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:41,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132996 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:43,003::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:39:43,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:43,003::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:43,004::__init__::232::IOProcess::(_processLogs) (7462) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:39:43,004::__init__::232::IOProcess::(_processLogs) (7462) Queuing response >Thread-16::DEBUG::2014-08-11 15:39:43,005::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:43,009::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:43,079::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:43,082::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:39:43,086::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00061284 s, 742 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:43,090::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:43,091::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:43,092::__init__::232::IOProcess::(_processLogs) (7463) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:39:43,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:43,095::__init__::232::IOProcess::(_processLogs) (7464) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:43,095::__init__::232::IOProcess::(_processLogs) (7464) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:43,096::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:43,098::__init__::232::IOProcess::(_processLogs) (7465) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:43,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:43,104::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:43,105::__init__::232::IOProcess::(_processLogs) (7466) Queuing response >Dummy-33::DEBUG::2014-08-11 15:39:43,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:43,887::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131817 s, 77.7 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:39:44,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`5df7bea6-32f0-4c4a-9fa6-e8c1f2eb8059`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:44,133::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:39:44,134::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00061284', 'lastCheck': '1.0', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:44,134::task::1191::Storage.TaskManager.Task::(prepare) Task=`5df7bea6-32f0-4c4a-9fa6-e8c1f2eb8059`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00061284', 'lastCheck': '1.0', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:44,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`5df7bea6-32f0-4c4a-9fa6-e8c1f2eb8059`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:44,135::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:44,135::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:44,135::task::993::Storage.TaskManager.Task::(_decref) Task=`5df7bea6-32f0-4c4a-9fa6-e8c1f2eb8059`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:39:44,154::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:39:45,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:45,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131712 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:47,965::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-126::DEBUG::2014-08-11 15:39:47,984::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:39:47,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0230630-0188-489d-b832-b194ec59b9f8`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:47,986::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Dummy-33::DEBUG::2014-08-11 15:39:48,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0139294 s, 73.5 MB/s\n'; <rc> = 0 >Thread-126::INFO::2014-08-11 15:39:48,007::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:48,009::task::1191::Storage.TaskManager.Task::(prepare) Task=`e0230630-0188-489d-b832-b194ec59b9f8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:48,058::task::595::Storage.TaskManager.Task::(_updateState) Task=`e0230630-0188-489d-b832-b194ec59b9f8`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:48,059::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:48,059::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:48,059::task::993::Storage.TaskManager.Task::(_decref) Task=`e0230630-0188-489d-b832-b194ec59b9f8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:39:48,070::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:39:48,071::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9ee6b2f-6dce-4924-a293-7903e074ef87`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:39:48,071::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:39:48,072::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9cf3ff8a-c77b-4460-a869-69a6b35c300c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:39:48,072::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:39:48,073::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:39:48,073::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9cf3ff8a-c77b-4460-a869-69a6b35c300c`::Granted request >Thread-34::DEBUG::2014-08-11 15:39:48,073::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d9ee6b2f-6dce-4924-a293-7903e074ef87`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:39:48,074::task::993::Storage.TaskManager.Task::(_decref) Task=`d9ee6b2f-6dce-4924-a293-7903e074ef87`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:39:48,094::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:48,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:39:48,096::task::1191::Storage.TaskManager.Task::(prepare) Task=`d9ee6b2f-6dce-4924-a293-7903e074ef87`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:48,097::task::595::Storage.TaskManager.Task::(_updateState) Task=`d9ee6b2f-6dce-4924-a293-7903e074ef87`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:39:48,099::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:39:48,100::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:39:48,102::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:48,102::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:39:48,103::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:39:48,104::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:39:48,104::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:39:48,106::task::993::Storage.TaskManager.Task::(_decref) Task=`d9ee6b2f-6dce-4924-a293-7903e074ef87`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:48,110::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:39:50,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:50,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133209 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:52,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:52,113::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131683 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:53,107::__init__::232::IOProcess::(_processLogs) (7467) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:39:53,108::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:53,113::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:53,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:53,179::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:39:53,181::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000645547 s, 705 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:39:53,182::__init__::232::IOProcess::(_processLogs) (7468) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:39:53,183::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:53,183::__init__::232::IOProcess::(_processLogs) (7469) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:53,184::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7469) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:53,184::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:39:53,185::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:53,185::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:53,185::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7470) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:53,185::__init__::232::IOProcess::(_processLogs) (7470) Queuing response >Thread-14::DEBUG::2014-08-11 15:39:53,186::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:39:53,187::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:39:53,187::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:39:53,187::__init__::232::IOProcess::(_processLogs) (7471) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:39:53,187::__init__::232::IOProcess::(_processLogs) (7471) Queuing response >Dummy-33::DEBUG::2014-08-11 15:39:54,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:54,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132383 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:39:56,196::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:56,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133038 s, 77.0 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:39:58,121::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:39:58,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c6bdd18-b5c4-45a8-b92a-e5933590ccb3`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:58,158::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:39:58,171::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:58,172::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c6bdd18-b5c4-45a8-b92a-e5933590ccb3`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:39:58,172::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c6bdd18-b5c4-45a8-b92a-e5933590ccb3`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:58,172::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:58,172::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:58,172::task::993::Storage.TaskManager.Task::(_decref) Task=`1c6bdd18-b5c4-45a8-b92a-e5933590ccb3`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:39:58,183::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:39:58,184::task::595::Storage.TaskManager.Task::(_updateState) Task=`b76e823b-0bcd-471d-8c11-d00494fe1ed5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:39:58,184::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:58,187::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:39:58,187::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a2d8da1a-a613-48de-a559-f3b85e08a3cc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:39:58,189::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:39:58,191::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:58,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:39:58,193::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a2d8da1a-a613-48de-a559-f3b85e08a3cc`::Granted request >Thread-34::DEBUG::2014-08-11 15:39:58,196::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b76e823b-0bcd-471d-8c11-d00494fe1ed5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:39:58,198::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:39:58,198::task::993::Storage.TaskManager.Task::(_decref) Task=`b76e823b-0bcd-471d-8c11-d00494fe1ed5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:39:58,221::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:58,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`b76e823b-0bcd-471d-8c11-d00494fe1ed5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:39:58,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`b76e823b-0bcd-471d-8c11-d00494fe1ed5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:39:58,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:39:58,222::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:39:58,223::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:39:58,223::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:39:58,223::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:39:58,223::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:39:58,224::task::993::Storage.TaskManager.Task::(_decref) Task=`b76e823b-0bcd-471d-8c11-d00494fe1ed5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:39:58,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:39:58,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132596 s, 77.2 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:39:59,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`6954cd79-eb13-4176-8f92-91c7958c495d`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:39:59,246::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:39:59,247::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000645547', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:59,247::task::1191::Storage.TaskManager.Task::(prepare) Task=`6954cd79-eb13-4176-8f92-91c7958c495d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000645547', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:39:59,247::task::595::Storage.TaskManager.Task::(_updateState) Task=`6954cd79-eb13-4176-8f92-91c7958c495d`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:39:59,247::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:39:59,247::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:39:59,248::task::993::Storage.TaskManager.Task::(_decref) Task=`6954cd79-eb13-4176-8f92-91c7958c495d`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:39:59,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:40:00,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:00,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185051 s, 55.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:02,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:02,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0265105 s, 38.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:03,189::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:03,190::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:03,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:03,190::__init__::232::IOProcess::(_processLogs) (7472) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:03,190::__init__::232::IOProcess::(_processLogs) (7472) Queuing response >Thread-16::DEBUG::2014-08-11 15:40:03,191::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:03,210::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:03,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:40:03,213::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00245979 s, 185 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:03,216::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:03,217::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:03,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:03,220::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:03,220::__init__::232::IOProcess::(_processLogs) (7473) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:03,221::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7473) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:03,222::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:03,223::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:03,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:03,224::__init__::232::IOProcess::(_processLogs) (7474) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:03,224::__init__::232::IOProcess::(_processLogs) (7474) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:03,225::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:03,225::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:03,226::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:03,226::__init__::232::IOProcess::(_processLogs) (7475) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:03,226::__init__::232::IOProcess::(_processLogs) (7475) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:03,227::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:03,227::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:03,228::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:03,229::__init__::232::IOProcess::(_processLogs) (7476) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:03,229::__init__::232::IOProcess::(_processLogs) (7476) Queuing response >Dummy-33::DEBUG::2014-08-11 15:40:04,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:04,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132012 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:06,751::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:06,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133685 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:08,226::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:08,228::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:08,229::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:40:08,236::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:40:08,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`290a375d-06bd-44ac-897f-e853355297e1`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:08,237::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:40:08,250::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:08,251::task::1191::Storage.TaskManager.Task::(prepare) Task=`290a375d-06bd-44ac-897f-e853355297e1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:08,251::task::595::Storage.TaskManager.Task::(_updateState) Task=`290a375d-06bd-44ac-897f-e853355297e1`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:08,251::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:08,251::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:08,252::task::993::Storage.TaskManager.Task::(_decref) Task=`290a375d-06bd-44ac-897f-e853355297e1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:40:08,263::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:40:08,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ad9943c-9c2d-4d2c-8f2d-abe3011ba9af`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:40:08,264::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:40:08,265::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a425de0-d8f5-4ff9-9eac-3e3eeb367232`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:40:08,265::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:40:08,265::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:40:08,266::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a425de0-d8f5-4ff9-9eac-3e3eeb367232`::Granted request >Thread-34::DEBUG::2014-08-11 15:40:08,266::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3ad9943c-9c2d-4d2c-8f2d-abe3011ba9af`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:40:08,266::task::993::Storage.TaskManager.Task::(_decref) Task=`3ad9943c-9c2d-4d2c-8f2d-abe3011ba9af`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:40:08,286::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:08,287::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ad9943c-9c2d-4d2c-8f2d-abe3011ba9af`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:08,287::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ad9943c-9c2d-4d2c-8f2d-abe3011ba9af`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:40:08,287::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:40:08,287::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:40:08,288::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:40:08,288::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:40:08,288::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:40:08,288::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:40:08,289::task::993::Storage.TaskManager.Task::(_decref) Task=`3ad9943c-9c2d-4d2c-8f2d-abe3011ba9af`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:40:08,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:08,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01328 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:10,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:10,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132664 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:12,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:12,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131442 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:13,230::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:13,231::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:40:13,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:40:13,233::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:13,253::__init__::232::IOProcess::(_processLogs) (7477) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:13,255::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:40:13,256::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000740038 s, 615 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:13,257::__init__::232::IOProcess::(_processLogs) (7477) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:13,260::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:13,261::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:13,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:13,264::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:13,265::__init__::232::IOProcess::(_processLogs) (7478) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:13,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:13,268::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7478) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:13,271::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:13,271::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:13,272::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:13,272::__init__::232::IOProcess::(_processLogs) (7479) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:13,272::__init__::232::IOProcess::(_processLogs) (7479) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:13,273::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:13,273::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:13,274::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:13,274::__init__::232::IOProcess::(_processLogs) (7480) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:13,275::__init__::232::IOProcess::(_processLogs) (7480) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:13,275::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:13,275::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:13,276::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:13,277::__init__::232::IOProcess::(_processLogs) (7481) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:13,277::__init__::232::IOProcess::(_processLogs) (7481) Queuing response >Thread-126::DEBUG::2014-08-11 15:40:14,367::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd7a803a-d6b0-4561-b006-60381ec532b1`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:14,368::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:40:14,368::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000740038', 'lastCheck': '1.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:14,368::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd7a803a-d6b0-4561-b006-60381ec532b1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000740038', 'lastCheck': '1.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:14,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd7a803a-d6b0-4561-b006-60381ec532b1`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:14,369::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:14,369::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:14,369::task::993::Storage.TaskManager.Task::(_decref) Task=`dd7a803a-d6b0-4561-b006-60381ec532b1`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:40:14,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:40:14,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:15,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132234 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:17,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:17,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132191 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:18,277::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:18,278::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:40:18,302::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-126::DEBUG::2014-08-11 15:40:18,363::task::595::Storage.TaskManager.Task::(_updateState) Task=`88aeb93d-1950-42d2-b574-53a796b15191`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:18,365::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:18,365::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:40:18,379::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:18,379::task::1191::Storage.TaskManager.Task::(prepare) Task=`88aeb93d-1950-42d2-b574-53a796b15191`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:18,380::task::595::Storage.TaskManager.Task::(_updateState) Task=`88aeb93d-1950-42d2-b574-53a796b15191`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:18,380::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:18,380::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:18,380::task::993::Storage.TaskManager.Task::(_decref) Task=`88aeb93d-1950-42d2-b574-53a796b15191`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:40:18,391::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-34::DEBUG::2014-08-11 15:40:18,392::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f3bbd47-ac8f-4c8f-8147-ea77f1d26110`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:40:18,392::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:40:18,393::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`974b57a5-3db1-444f-b59b-c756a777b613`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:40:18,393::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:40:18,394::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:40:18,394::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`974b57a5-3db1-444f-b59b-c756a777b613`::Granted request >Thread-34::DEBUG::2014-08-11 15:40:18,394::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4f3bbd47-ac8f-4c8f-8147-ea77f1d26110`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:40:18,395::task::993::Storage.TaskManager.Task::(_decref) Task=`4f3bbd47-ac8f-4c8f-8147-ea77f1d26110`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:40:18,414::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:18,415::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f3bbd47-ac8f-4c8f-8147-ea77f1d26110`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:18,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f3bbd47-ac8f-4c8f-8147-ea77f1d26110`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:40:18,415::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:40:18,415::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:40:18,416::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:40:18,416::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:40:18,416::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:40:18,417::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:40:18,417::task::993::Storage.TaskManager.Task::(_decref) Task=`4f3bbd47-ac8f-4c8f-8147-ea77f1d26110`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:40:19,085::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:19,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133487 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:21,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:21,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133099 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:23,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:23,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0144622 s, 70.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:23,277::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:23,278::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:40:23,279::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:40:23,361::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:23,380::__init__::232::IOProcess::(_processLogs) (7482) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:23,382::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:23,383::__init__::232::IOProcess::(_processLogs) (7482) Queuing response >Thread-16::DEBUG::2014-08-11 15:40:23,384::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00075176 s, 605 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:23,387::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:23,388::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:23,391::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:23,393::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:23,391::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:23,394::__init__::232::IOProcess::(_processLogs) (7483) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:23,395::__init__::232::IOProcess::(_processLogs) (7483) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:23,396::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:23,396::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:23,397::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:23,397::__init__::232::IOProcess::(_processLogs) (7484) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:23,397::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7484) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:23,398::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:23,399::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:23,399::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:23,400::__init__::232::IOProcess::(_processLogs) (7485) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:23,400::__init__::232::IOProcess::(_processLogs) (7485) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:23,400::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:23,401::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:23,401::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:23,401::__init__::232::IOProcess::(_processLogs) (7486) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:23,402::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7486) Queuing response >Thread-122::DEBUG::2014-08-11 15:40:24,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`56c7b55c-1c9d-40e1-ac21-eeadd6d38a6c`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:40:24,904::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:40:24,908::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:24,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:24,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:24,909::__init__::232::IOProcess::(_processLogs) (7487) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:40:24,911::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7487) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:24,912::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:24,913::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:24,913::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:24,914::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7488) Got request for method 'stat' >Thread-122::INFO::2014-08-11 15:40:24,914::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:40:24,915::__init__::232::IOProcess::(_processLogs) (7488) Queuing response >Thread-122::DEBUG::2014-08-11 15:40:24,916::task::1191::Storage.TaskManager.Task::(prepare) Task=`56c7b55c-1c9d-40e1-ac21-eeadd6d38a6c`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:40:24,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`56c7b55c-1c9d-40e1-ac21-eeadd6d38a6c`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:40:24,917::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:40:24,917::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:40:24,918::task::993::Storage.TaskManager.Task::(_decref) Task=`56c7b55c-1c9d-40e1-ac21-eeadd6d38a6c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:40:25,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:25,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131533 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:27,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:27,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132323 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:28,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:28,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:28,403::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:40:28,429::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:40:28,430::task::595::Storage.TaskManager.Task::(_updateState) Task=`550573c7-8035-497c-9927-69e8dec9224f`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:28,430::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:40:28,443::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:28,443::task::1191::Storage.TaskManager.Task::(prepare) Task=`550573c7-8035-497c-9927-69e8dec9224f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:28,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`550573c7-8035-497c-9927-69e8dec9224f`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:28,444::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:28,444::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:28,444::task::993::Storage.TaskManager.Task::(_decref) Task=`550573c7-8035-497c-9927-69e8dec9224f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:40:28,461::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:40:28,462::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7851842-2161-4ccb-8a50-269522e0ce8f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:40:28,462::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:40:28,463::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f082160b-4006-4dd4-9835-922730fb45fa`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:40:28,463::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:40:28,463::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:40:28,464::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f082160b-4006-4dd4-9835-922730fb45fa`::Granted request >Thread-34::DEBUG::2014-08-11 15:40:28,464::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d7851842-2161-4ccb-8a50-269522e0ce8f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:40:28,464::task::993::Storage.TaskManager.Task::(_decref) Task=`d7851842-2161-4ccb-8a50-269522e0ce8f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:40:28,484::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:28,486::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7851842-2161-4ccb-8a50-269522e0ce8f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:28,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7851842-2161-4ccb-8a50-269522e0ce8f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:40:28,487::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:40:28,488::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:40:28,489::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:40:28,490::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:40:28,491::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:40:28,492::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:40:28,494::task::993::Storage.TaskManager.Task::(_decref) Task=`d7851842-2161-4ccb-8a50-269522e0ce8f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:40:29,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:29,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132203 s, 77.5 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:40:29,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`424f4b5e-b4b0-466d-8c57-a05dcf52c9c0`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:29,564::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:40:29,564::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00075176', 'lastCheck': '6.2', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:29,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`424f4b5e-b4b0-466d-8c57-a05dcf52c9c0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00075176', 'lastCheck': '6.2', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:29,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`424f4b5e-b4b0-466d-8c57-a05dcf52c9c0`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:29,565::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:29,565::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:29,566::task::993::Storage.TaskManager.Task::(_decref) Task=`424f4b5e-b4b0-466d-8c57-a05dcf52c9c0`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:40:29,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:40:31,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:31,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135109 s, 75.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:33,403::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:33,404::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:33,410::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:33,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:33,462::__init__::232::IOProcess::(_processLogs) (7489) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:33,464::__init__::232::IOProcess::(_processLogs) (7489) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:33,465::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:40:33,468::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:33,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:40:33,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 15:40:33,508::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000929384 s, 490 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:33,511::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:33,512::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:33,513::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:33,514::__init__::232::IOProcess::(_processLogs) (7490) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:33,519::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7490) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:33,524::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:33,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:33,524::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:33,525::__init__::232::IOProcess::(_processLogs) (7491) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:33,526::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7491) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:33,528::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:33,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:33,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:33,529::__init__::232::IOProcess::(_processLogs) (7492) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:33,529::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7492) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:33,531::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:33,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:33,532::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:33,532::__init__::232::IOProcess::(_processLogs) (7493) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:33,533::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(7493) Queuing response >Dummy-33::DEBUG::2014-08-11 15:40:33,538::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277681 s, 36.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:35,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:35,594::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131997 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:37,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:37,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013174 s, 77.7 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:40:38,507::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:40:38,508::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0dfd519-946f-4934-a8ae-078e430e026f`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:38,519::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:40:38,561::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:38,564::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:38,565::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:40:38,578::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:38,578::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0dfd519-946f-4934-a8ae-078e430e026f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:38,578::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0dfd519-946f-4934-a8ae-078e430e026f`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:38,578::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:38,579::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:38,579::task::993::Storage.TaskManager.Task::(_decref) Task=`d0dfd519-946f-4934-a8ae-078e430e026f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:40:38,589::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:40:38,590::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd2eafc1-2b9f-4ea2-9d2c-9aadbafad993`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:40:38,590::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:40:38,591::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4c894c3b-616a-497d-882c-3f53e2d363e3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:40:38,591::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:40:38,592::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:40:38,592::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4c894c3b-616a-497d-882c-3f53e2d363e3`::Granted request >Thread-34::DEBUG::2014-08-11 15:40:38,593::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bd2eafc1-2b9f-4ea2-9d2c-9aadbafad993`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:40:38,593::task::993::Storage.TaskManager.Task::(_decref) Task=`bd2eafc1-2b9f-4ea2-9d2c-9aadbafad993`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:40:38,612::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:38,613::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd2eafc1-2b9f-4ea2-9d2c-9aadbafad993`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952281088', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:38,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd2eafc1-2b9f-4ea2-9d2c-9aadbafad993`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:40:38,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:40:38,614::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:40:38,614::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:40:38,614::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:40:38,614::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:40:38,615::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:40:38,615::task::993::Storage.TaskManager.Task::(_decref) Task=`bd2eafc1-2b9f-4ea2-9d2c-9aadbafad993`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:40:39,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:39,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132084 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:41,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:41,761::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132768 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:43,536::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:43,537::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:40:43,538::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:40:43,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:43,581::__init__::232::IOProcess::(_processLogs) (7494) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:43,583::__init__::232::IOProcess::(_processLogs) (7494) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:43,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:40:43,585::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000574567 s, 792 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:43,589::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:43,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:43,591::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:43,592::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:43,592::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:43,592::__init__::232::IOProcess::(_processLogs) (7495) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:43,593::__init__::232::IOProcess::(_processLogs) DEBUG|(7495) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:43,594::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:43,595::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:43,595::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:43,596::__init__::232::IOProcess::(_processLogs) (7496) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:43,596::__init__::232::IOProcess::(_processLogs) (7496) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:43,596::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:43,597::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:43,597::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:43,597::__init__::232::IOProcess::(_processLogs) (7497) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:43,598::__init__::232::IOProcess::(_processLogs) DEBUG|(7497) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:43,598::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:43,599::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:43,600::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:43,601::__init__::232::IOProcess::(_processLogs) (7498) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:43,601::__init__::232::IOProcess::(_processLogs) (7498) Queuing response >Dummy-33::DEBUG::2014-08-11 15:40:43,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:43,817::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133409 s, 76.8 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:40:44,677::task::595::Storage.TaskManager.Task::(_updateState) Task=`22c572c2-7ac7-434b-a339-72254a61fa14`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:44,677::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:40:44,678::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000574567', 'lastCheck': '1.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:44,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`22c572c2-7ac7-434b-a339-72254a61fa14`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000574567', 'lastCheck': '1.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:44,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`22c572c2-7ac7-434b-a339-72254a61fa14`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:44,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:44,678::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:44,679::task::993::Storage.TaskManager.Task::(_decref) Task=`22c572c2-7ac7-434b-a339-72254a61fa14`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:40:44,700::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:40:45,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:45,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131569 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:47,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:47,928::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132506 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:48,597::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:48,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:40:48,627::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:40:48,664::task::595::Storage.TaskManager.Task::(_updateState) Task=`c399203b-ee29-4d0b-afc2-eaee13438d69`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:48,666::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:40:48,666::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:40:48,680::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:48,680::task::1191::Storage.TaskManager.Task::(prepare) Task=`c399203b-ee29-4d0b-afc2-eaee13438d69`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:48,681::task::595::Storage.TaskManager.Task::(_updateState) Task=`c399203b-ee29-4d0b-afc2-eaee13438d69`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:48,681::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:48,681::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:48,681::task::993::Storage.TaskManager.Task::(_decref) Task=`c399203b-ee29-4d0b-afc2-eaee13438d69`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:40:48,692::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:40:48,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`02c57ed4-0eb9-450f-adbf-ce11e2206ba8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:40:48,693::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:40:48,694::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5063d10f-7a94-476c-a741-0895301cce26`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:40:48,694::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:40:48,694::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:40:48,695::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5063d10f-7a94-476c-a741-0895301cce26`::Granted request >Thread-34::DEBUG::2014-08-11 15:40:48,695::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`02c57ed4-0eb9-450f-adbf-ce11e2206ba8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:40:48,695::task::993::Storage.TaskManager.Task::(_decref) Task=`02c57ed4-0eb9-450f-adbf-ce11e2206ba8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:40:48,715::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952215552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:48,715::task::1191::Storage.TaskManager.Task::(prepare) Task=`02c57ed4-0eb9-450f-adbf-ce11e2206ba8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952215552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:48,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`02c57ed4-0eb9-450f-adbf-ce11e2206ba8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:40:48,716::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:40:48,716::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:40:48,716::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:40:48,717::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:40:48,717::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:40:48,717::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:40:48,717::task::993::Storage.TaskManager.Task::(_decref) Task=`02c57ed4-0eb9-450f-adbf-ce11e2206ba8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:40:49,951::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:49,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133216 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:52,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:52,039::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013158 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:53,601::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:53,602::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:40:53,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:40:53,662::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:53,681::__init__::232::IOProcess::(_processLogs) (7499) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:53,682::__init__::232::IOProcess::(_processLogs) (7499) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:53,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:40:53,685::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00103022 s, 442 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:40:53,688::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:53,690::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:53,692::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:53,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:53,693::__init__::232::IOProcess::(_processLogs) (7500) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:40:53,694::__init__::232::IOProcess::(_processLogs) (7500) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:53,697::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:40:53,697::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:53,699::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:53,700::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:53,701::__init__::232::IOProcess::(_processLogs) (7501) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:53,701::__init__::232::IOProcess::(_processLogs) (7501) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:53,702::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:53,702::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:53,703::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:53,703::__init__::232::IOProcess::(_processLogs) (7502) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:53,703::__init__::232::IOProcess::(_processLogs) DEBUG|(7502) Queuing response >Thread-14::DEBUG::2014-08-11 15:40:53,704::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:40:53,705::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:40:53,705::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:40:53,705::__init__::232::IOProcess::(_processLogs) (7503) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:40:53,706::__init__::232::IOProcess::(_processLogs) DEBUG|(7503) Queuing response >Dummy-33::DEBUG::2014-08-11 15:40:54,062::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:54,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132546 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:56,118::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:56,151::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133712 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:40:58,174::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:40:58,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013237 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:58,705::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:58,706::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:40:58,730::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:40:58,764::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2c04135-9226-4215-8980-e78a366d94ac`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:58,765::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:40:58,767::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:40:58,780::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:58,780::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2c04135-9226-4215-8980-e78a366d94ac`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:40:58,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2c04135-9226-4215-8980-e78a366d94ac`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:58,780::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:58,781::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:58,781::task::993::Storage.TaskManager.Task::(_decref) Task=`f2c04135-9226-4215-8980-e78a366d94ac`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:40:58,793::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:40:58,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8fff363-8b28-4c21-aa43-d86e0b47fc3f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:40:58,794::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:40:58,795::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`af2e1799-f8f5-482f-9502-1b7a453deaae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:40:58,795::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:40:58,796::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:40:58,796::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`af2e1799-f8f5-482f-9502-1b7a453deaae`::Granted request >Thread-34::DEBUG::2014-08-11 15:40:58,796::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a8fff363-8b28-4c21-aa43-d86e0b47fc3f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:40:58,797::task::993::Storage.TaskManager.Task::(_decref) Task=`a8fff363-8b28-4c21-aa43-d86e0b47fc3f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:40:58,816::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952215552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:58,817::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8fff363-8b28-4c21-aa43-d86e0b47fc3f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952215552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:40:58,817::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8fff363-8b28-4c21-aa43-d86e0b47fc3f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:40:58,817::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:40:58,818::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:40:58,818::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:40:58,818::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:40:58,818::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:40:58,819::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:40:58,819::task::993::Storage.TaskManager.Task::(_decref) Task=`a8fff363-8b28-4c21-aa43-d86e0b47fc3f`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:40:59,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`82028b4d-82d9-4c49-a483-3a2dcf1b4a89`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:40:59,803::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:40:59,803::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00103022', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:59,803::task::1191::Storage.TaskManager.Task::(prepare) Task=`82028b4d-82d9-4c49-a483-3a2dcf1b4a89`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00103022', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:40:59,804::task::595::Storage.TaskManager.Task::(_updateState) Task=`82028b4d-82d9-4c49-a483-3a2dcf1b4a89`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:40:59,804::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:40:59,804::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:40:59,804::task::993::Storage.TaskManager.Task::(_decref) Task=`82028b4d-82d9-4c49-a483-3a2dcf1b4a89`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:40:59,825::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:41:00,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:00,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131322 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:02,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:02,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.031705 s, 32.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:03,707::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:03,708::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:03,762::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:03,763::__init__::232::IOProcess::(_processLogs) (7504) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:41:03,763::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:41:03,764::__init__::232::IOProcess::(_processLogs) (7504) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:03,784::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:03,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:03,787::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:41:03,788::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00386873 s, 118 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:03,789::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:03,790::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:03,790::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:03,790::__init__::232::IOProcess::(_processLogs) (7505) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:03,790::__init__::232::IOProcess::(_processLogs) DEBUG|(7505) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:03,792::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:03,792::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:03,792::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:03,792::__init__::232::IOProcess::(_processLogs) (7506) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:03,793::__init__::232::IOProcess::(_processLogs) DEBUG|(7506) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:03,793::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:03,794::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:03,794::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:03,794::__init__::232::IOProcess::(_processLogs) (7507) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:03,795::__init__::232::IOProcess::(_processLogs) DEBUG|(7507) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:03,795::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:03,796::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:03,796::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:03,796::__init__::232::IOProcess::(_processLogs) (7508) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:03,796::__init__::232::IOProcess::(_processLogs) DEBUG|(7508) Queuing response >Dummy-33::DEBUG::2014-08-11 15:41:04,583::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:04,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132408 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:06,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:06,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132162 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:08,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:08,744::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132119 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:08,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:08,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:08,797::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:41:08,830::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:41:08,831::task::595::Storage.TaskManager.Task::(_updateState) Task=`af944799-dc81-45ef-87cd-0144a586a288`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:08,832::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:41:08,844::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:08,845::task::1191::Storage.TaskManager.Task::(prepare) Task=`af944799-dc81-45ef-87cd-0144a586a288`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:08,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`af944799-dc81-45ef-87cd-0144a586a288`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:08,845::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:08,845::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:08,846::task::993::Storage.TaskManager.Task::(_decref) Task=`af944799-dc81-45ef-87cd-0144a586a288`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:41:08,858::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:41:08,859::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f6feec9-13fc-4198-87aa-f38300fa20f2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:41:08,859::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:41:08,860::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0ffe849f-d054-4e60-8fb9-a0f73c53ee25`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:41:08,860::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:41:08,860::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:41:08,860::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0ffe849f-d054-4e60-8fb9-a0f73c53ee25`::Granted request >Thread-34::DEBUG::2014-08-11 15:41:08,861::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5f6feec9-13fc-4198-87aa-f38300fa20f2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:41:08,861::task::993::Storage.TaskManager.Task::(_decref) Task=`5f6feec9-13fc-4198-87aa-f38300fa20f2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:41:08,881::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952215552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:08,881::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f6feec9-13fc-4198-87aa-f38300fa20f2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952215552', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:08,882::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f6feec9-13fc-4198-87aa-f38300fa20f2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:41:08,882::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:41:08,882::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:41:08,882::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:41:08,883::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:41:08,883::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:41:08,883::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:41:08,884::task::993::Storage.TaskManager.Task::(_decref) Task=`5f6feec9-13fc-4198-87aa-f38300fa20f2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:41:10,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:10,818::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131001 s, 78.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:12,840::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:12,872::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133935 s, 76.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:13,799::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:13,803::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:41:13,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:13,863::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:13,866::__init__::232::IOProcess::(_processLogs) DEBUG|(7509) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:41:13,867::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:13,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:41:13,887::__init__::232::IOProcess::(_processLogs) (7509) Queuing response >Thread-16::DEBUG::2014-08-11 15:41:13,890::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000602557 s, 755 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:13,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:41:13,894::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:13,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:13,896::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:13,896::__init__::232::IOProcess::(_processLogs) (7510) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:13,897::__init__::232::IOProcess::(_processLogs) DEBUG|(7510) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:13,898::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:13,898::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:13,899::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:13,899::__init__::232::IOProcess::(_processLogs) (7511) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:13,900::__init__::232::IOProcess::(_processLogs) DEBUG|(7511) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:13,900::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:13,900::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:13,901::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:13,901::__init__::232::IOProcess::(_processLogs) (7512) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:13,902::__init__::232::IOProcess::(_processLogs) DEBUG|(7512) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:13,902::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:13,903::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:13,903::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:13,903::__init__::232::IOProcess::(_processLogs) (7513) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:13,904::__init__::232::IOProcess::(_processLogs) DEBUG|(7513) Queuing response >Dummy-33::DEBUG::2014-08-11 15:41:14,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-126::DEBUG::2014-08-11 15:41:14,922::task::595::Storage.TaskManager.Task::(_updateState) Task=`0872cd1b-793a-4395-9b49-5a00f9eda3dc`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:14,922::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:41:14,923::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000602557', 'lastCheck': '1.0', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:41:14,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`0872cd1b-793a-4395-9b49-5a00f9eda3dc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000602557', 'lastCheck': '1.0', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:41:14,925::task::595::Storage.TaskManager.Task::(_updateState) Task=`0872cd1b-793a-4395-9b49-5a00f9eda3dc`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:14,926::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:14,927::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:14,928::task::993::Storage.TaskManager.Task::(_decref) Task=`0872cd1b-793a-4395-9b49-5a00f9eda3dc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:41:14,947::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0305864 s, 33.5 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:41:14,968::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:41:16,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:17,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131228 s, 78.0 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:41:18,898::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:41:18,900::task::595::Storage.TaskManager.Task::(_updateState) Task=`f814ac65-9884-4d2a-ae9b-197850b62c68`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:18,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:41:18,962::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:18,965::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:18,966::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:41:18,978::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:18,978::task::1191::Storage.TaskManager.Task::(prepare) Task=`f814ac65-9884-4d2a-ae9b-197850b62c68`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:18,979::task::595::Storage.TaskManager.Task::(_updateState) Task=`f814ac65-9884-4d2a-ae9b-197850b62c68`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:18,979::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:18,979::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:18,979::task::993::Storage.TaskManager.Task::(_decref) Task=`f814ac65-9884-4d2a-ae9b-197850b62c68`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:41:18,990::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:41:18,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e40ef8e-25b6-492f-a5d7-288c4f35c3ca`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:41:18,991::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:41:18,991::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d75dcbe8-7412-4ea3-b490-3f060b0cf8c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:41:18,992::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:41:18,992::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:41:18,992::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d75dcbe8-7412-4ea3-b490-3f060b0cf8c8`::Granted request >Thread-34::DEBUG::2014-08-11 15:41:18,993::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6e40ef8e-25b6-492f-a5d7-288c4f35c3ca`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:41:18,993::task::993::Storage.TaskManager.Task::(_decref) Task=`6e40ef8e-25b6-492f-a5d7-288c4f35c3ca`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:41:19,013::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:19,013::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e40ef8e-25b6-492f-a5d7-288c4f35c3ca`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:19,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e40ef8e-25b6-492f-a5d7-288c4f35c3ca`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:41:19,014::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:41:19,014::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:41:19,014::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:41:19,015::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:41:19,015::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:41:19,015::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:41:19,015::task::993::Storage.TaskManager.Task::(_decref) Task=`6e40ef8e-25b6-492f-a5d7-288c4f35c3ca`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:41:19,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:19,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132518 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:21,100::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:21,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133234 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:23,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:23,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013237 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:23,905::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:23,906::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:23,906::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:23,906::__init__::232::IOProcess::(_processLogs) (7514) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:23,907::__init__::232::IOProcess::(_processLogs) DEBUG|(7514) Queuing response >Thread-16::DEBUG::2014-08-11 15:41:23,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:41:23,927::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416698 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:23,928::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:23,929::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:23,929::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:23,929::__init__::232::IOProcess::(_processLogs) (7515) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:23,930::__init__::232::IOProcess::(_processLogs) DEBUG|(7515) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:23,931::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:23,931::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:23,932::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:23,932::__init__::232::IOProcess::(_processLogs) DEBUG|(7516) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:23,932::__init__::232::IOProcess::(_processLogs) (7516) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:23,933::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:23,933::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:23,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:23,934::__init__::232::IOProcess::(_processLogs) (7517) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:23,934::__init__::232::IOProcess::(_processLogs) (7517) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:23,935::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:23,935::__init__::232::IOProcess::(_processLogs) (7518) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:23,970::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:23,972::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:23,972::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-122::DEBUG::2014-08-11 15:41:25,047::task::595::Storage.TaskManager.Task::(_updateState) Task=`0771ef7c-9ed8-460e-bd48-0da07fb730e0`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:41:25,047::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:41:25,048::__init__::232::IOProcess::(_processLogs) (7519) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:41:25,050::__init__::232::IOProcess::(_processLogs) (7520) Got request for method 'stat' >Thread-122::INFO::2014-08-11 15:41:25,051::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:41:25,051::task::1191::Storage.TaskManager.Task::(prepare) Task=`0771ef7c-9ed8-460e-bd48-0da07fb730e0`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:41:25,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`0771ef7c-9ed8-460e-bd48-0da07fb730e0`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:41:25,052::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:41:25,052::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:41:25,052::task::993::Storage.TaskManager.Task::(_decref) Task=`0771ef7c-9ed8-460e-bd48-0da07fb730e0`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:41:25,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:25,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132629 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:27,266::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:27,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013306 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:28,976::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:28,977::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:28,979::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:41:29,029::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:41:29,030::task::595::Storage.TaskManager.Task::(_updateState) Task=`549b2187-1b87-4092-acf3-459dd4db2494`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:29,030::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:41:29,043::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:29,044::task::1191::Storage.TaskManager.Task::(prepare) Task=`549b2187-1b87-4092-acf3-459dd4db2494`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:29,044::task::595::Storage.TaskManager.Task::(_updateState) Task=`549b2187-1b87-4092-acf3-459dd4db2494`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:29,044::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:29,044::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:29,045::task::993::Storage.TaskManager.Task::(_decref) Task=`549b2187-1b87-4092-acf3-459dd4db2494`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:41:29,056::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:41:29,057::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e0b14f5-2d55-4b56-8bce-ea0a38a813a3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:41:29,057::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:41:29,058::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ece6a5d9-4e9b-46dd-b9e3-f39e14e29b1e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:41:29,058::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:41:29,059::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:41:29,059::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ece6a5d9-4e9b-46dd-b9e3-f39e14e29b1e`::Granted request >Thread-34::DEBUG::2014-08-11 15:41:29,059::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6e0b14f5-2d55-4b56-8bce-ea0a38a813a3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:41:29,060::task::993::Storage.TaskManager.Task::(_decref) Task=`6e0b14f5-2d55-4b56-8bce-ea0a38a813a3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:41:29,079::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:29,080::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e0b14f5-2d55-4b56-8bce-ea0a38a813a3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:29,080::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e0b14f5-2d55-4b56-8bce-ea0a38a813a3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:41:29,080::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:41:29,080::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:41:29,081::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:41:29,081::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:41:29,082::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:41:29,082::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:41:29,082::task::993::Storage.TaskManager.Task::(_decref) Task=`6e0b14f5-2d55-4b56-8bce-ea0a38a813a3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:41:29,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:29,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131767 s, 77.7 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:41:30,072::task::595::Storage.TaskManager.Task::(_updateState) Task=`40f219c5-d6da-49fb-a547-506931a3ec6d`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:30,073::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:41:30,073::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416698', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:41:30,074::task::1191::Storage.TaskManager.Task::(prepare) Task=`40f219c5-d6da-49fb-a547-506931a3ec6d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416698', 'lastCheck': '6.1', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:41:30,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`40f219c5-d6da-49fb-a547-506931a3ec6d`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:30,074::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:30,074::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:30,074::task::993::Storage.TaskManager.Task::(_decref) Task=`40f219c5-d6da-49fb-a547-506931a3ec6d`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:41:30,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:41:31,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:31,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133378 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:33,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:33,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131602 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:33,937::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:33,937::__init__::232::IOProcess::(_processLogs) (7521) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:41:33,938::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:41:33,957::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000428462 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:33,959::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:33,959::__init__::232::IOProcess::(_processLogs) (7522) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:33,960::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:33,961::__init__::232::IOProcess::(_processLogs) (7523) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:33,961::__init__::232::IOProcess::(_processLogs) DEBUG|(7523) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:33,962::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:33,962::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:33,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:33,962::__init__::232::IOProcess::(_processLogs) DEBUG|(7524) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:33,963::__init__::232::IOProcess::(_processLogs) (7524) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:33,963::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:33,964::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:33,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:33,965::__init__::232::IOProcess::(_processLogs) DEBUG|(7525) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:33,965::__init__::232::IOProcess::(_processLogs) (7525) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:33,985::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:33,986::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:34,062::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:41:35,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:35,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133123 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:37,549::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:37,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132231 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:39,069::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:39,070::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:39,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:41:39,097::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:41:39,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d88007a-a5a3-4feb-93b8-31c8698a01b1`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:39,098::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:41:39,111::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:39,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d88007a-a5a3-4feb-93b8-31c8698a01b1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:39,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d88007a-a5a3-4feb-93b8-31c8698a01b1`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:39,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:39,113::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:39,113::task::993::Storage.TaskManager.Task::(_decref) Task=`7d88007a-a5a3-4feb-93b8-31c8698a01b1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:41:39,122::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:41:39,123::task::595::Storage.TaskManager.Task::(_updateState) Task=`a955ec55-6795-4468-814f-f2d9e2157245`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:41:39,123::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:41:39,124::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b95d2c53-36ce-4b9f-9e3f-68387300f9f6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:41:39,124::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:41:39,125::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:41:39,125::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b95d2c53-36ce-4b9f-9e3f-68387300f9f6`::Granted request >Thread-34::DEBUG::2014-08-11 15:41:39,125::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a955ec55-6795-4468-814f-f2d9e2157245`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:41:39,126::task::993::Storage.TaskManager.Task::(_decref) Task=`a955ec55-6795-4468-814f-f2d9e2157245`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:41:39,146::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:39,146::task::1191::Storage.TaskManager.Task::(prepare) Task=`a955ec55-6795-4468-814f-f2d9e2157245`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:39,146::task::595::Storage.TaskManager.Task::(_updateState) Task=`a955ec55-6795-4468-814f-f2d9e2157245`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:41:39,147::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:41:39,147::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:41:39,147::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:41:39,148::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:41:39,148::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:41:39,148::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:41:39,148::task::993::Storage.TaskManager.Task::(_decref) Task=`a955ec55-6795-4468-814f-f2d9e2157245`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:41:39,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:39,637::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133402 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:41,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:41,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131237 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:43,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:43,752::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141388 s, 72.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:43,967::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:43,967::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:43,968::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:43,968::__init__::232::IOProcess::(_processLogs) (7526) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:43,968::__init__::232::IOProcess::(_processLogs) (7526) Queuing response >Thread-16::DEBUG::2014-08-11 15:41:43,969::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:41:43,988::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415464 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:43,989::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:43,989::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:43,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:43,990::__init__::232::IOProcess::(_processLogs) (7527) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:43,990::__init__::232::IOProcess::(_processLogs) (7527) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:43,991::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:43,992::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:43,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:43,993::__init__::232::IOProcess::(_processLogs) (7528) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:43,993::__init__::232::IOProcess::(_processLogs) (7528) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:43,993::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:43,994::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:43,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:43,994::__init__::232::IOProcess::(_processLogs) (7529) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:43,994::__init__::232::IOProcess::(_processLogs) DEBUG|(7529) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:43,995::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:43,996::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:43,997::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:43,998::__init__::232::IOProcess::(_processLogs) (7530) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:43,998::__init__::232::IOProcess::(_processLogs) (7530) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:44,080::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:44,082::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:44,162::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:41:45,200::task::595::Storage.TaskManager.Task::(_updateState) Task=`9235bae2-8cb7-4f2c-a460-fa43b31cfb5a`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:45,200::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:41:45,200::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415464', 'lastCheck': '1.2', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:41:45,201::task::1191::Storage.TaskManager.Task::(prepare) Task=`9235bae2-8cb7-4f2c-a460-fa43b31cfb5a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000415464', 'lastCheck': '1.2', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:41:45,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`9235bae2-8cb7-4f2c-a460-fa43b31cfb5a`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:45,201::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:45,201::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:45,202::task::993::Storage.TaskManager.Task::(_decref) Task=`9235bae2-8cb7-4f2c-a460-fa43b31cfb5a`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:41:45,221::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:41:45,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:45,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131892 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:47,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:47,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132655 s, 77.2 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:41:49,164::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:41:49,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8797706-1709-4e9a-86fb-4c05542507e9`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:49,262::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:49,268::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:49,270::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:49,271::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:41:49,274::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:49,275::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8797706-1709-4e9a-86fb-4c05542507e9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:49,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8797706-1709-4e9a-86fb-4c05542507e9`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:49,275::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:49,275::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:49,276::task::993::Storage.TaskManager.Task::(_decref) Task=`c8797706-1709-4e9a-86fb-4c05542507e9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:41:49,285::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:41:49,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`eef611c0-7030-488c-9467-5d8093926220`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:41:49,286::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:41:49,287::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`014a69e1-2b28-480b-8697-10589c5e56eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:41:49,287::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:41:49,288::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:41:49,288::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`014a69e1-2b28-480b-8697-10589c5e56eb`::Granted request >Thread-34::DEBUG::2014-08-11 15:41:49,288::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`eef611c0-7030-488c-9467-5d8093926220`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:41:49,289::task::993::Storage.TaskManager.Task::(_decref) Task=`eef611c0-7030-488c-9467-5d8093926220`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:41:49,315::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:49,315::task::1191::Storage.TaskManager.Task::(prepare) Task=`eef611c0-7030-488c-9467-5d8093926220`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:49,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`eef611c0-7030-488c-9467-5d8093926220`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:41:49,316::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:41:49,316::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:41:49,316::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:41:49,316::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:41:49,317::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:41:49,317::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:41:49,317::task::993::Storage.TaskManager.Task::(_decref) Task=`eef611c0-7030-488c-9467-5d8093926220`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:41:49,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:49,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131556 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:51,940::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:51,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130806 s, 78.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:53,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:41:54,014::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:54,015::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:54,016::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:54,016::__init__::232::IOProcess::(_processLogs) (7531) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:41:54,018::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:41:54,018::__init__::232::IOProcess::(_processLogs) DEBUG|(7531) Queuing response >Dummy-33::DEBUG::2014-08-11 15:41:54,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283838 s, 36.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:41:54,045::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00465952 s, 97.6 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:41:54,048::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:54,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:54,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:54,052::__init__::232::IOProcess::(_processLogs) (7532) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:41:54,054::__init__::232::IOProcess::(_processLogs) DEBUG|(7532) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:54,057::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:54,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:54,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:54,060::__init__::232::IOProcess::(_processLogs) (7533) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:54,061::__init__::232::IOProcess::(_processLogs) (7533) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:54,062::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:54,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:54,064::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:54,065::__init__::232::IOProcess::(_processLogs) (7534) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:54,066::__init__::232::IOProcess::(_processLogs) (7534) Queuing response >Thread-14::DEBUG::2014-08-11 15:41:54,068::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:41:54,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:41:54,071::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:41:54,072::__init__::232::IOProcess::(_processLogs) (7535) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:41:54,073::__init__::232::IOProcess::(_processLogs) (7535) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:54,275::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:54,277::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:54,362::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:41:56,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:56,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133013 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:41:58,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:41:58,324::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132891 s, 77.1 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:41:59,330::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:41:59,331::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ce9c9e7-d034-4acc-abc2-17146a9be371`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:41:59,363::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:59,370::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:59,371::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:41:59,372::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:41:59,376::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:59,376::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ce9c9e7-d034-4acc-abc2-17146a9be371`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:41:59,377::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ce9c9e7-d034-4acc-abc2-17146a9be371`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:41:59,377::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:41:59,377::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:41:59,377::task::993::Storage.TaskManager.Task::(_decref) Task=`0ce9c9e7-d034-4acc-abc2-17146a9be371`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:41:59,388::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:41:59,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`f31f6e19-256f-4342-9802-1c989b8ab9df`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:41:59,389::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:41:59,389::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9ecda7c2-3004-4220-a190-2c893d04d729`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:41:59,390::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:41:59,390::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:41:59,390::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9ecda7c2-3004-4220-a190-2c893d04d729`::Granted request >Thread-34::DEBUG::2014-08-11 15:41:59,391::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f31f6e19-256f-4342-9802-1c989b8ab9df`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:41:59,391::task::993::Storage.TaskManager.Task::(_decref) Task=`f31f6e19-256f-4342-9802-1c989b8ab9df`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:41:59,411::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:59,411::task::1191::Storage.TaskManager.Task::(prepare) Task=`f31f6e19-256f-4342-9802-1c989b8ab9df`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:41:59,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`f31f6e19-256f-4342-9802-1c989b8ab9df`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:41:59,412::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:41:59,412::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:41:59,412::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:41:59,413::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:41:59,413::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:41:59,413::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:41:59,414::task::993::Storage.TaskManager.Task::(_decref) Task=`f31f6e19-256f-4342-9802-1c989b8ab9df`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:42:00,314::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9d285f4-634f-4559-b4bb-384db438b79b`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:00,314::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:42:00,315::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00465952', 'lastCheck': '6.2', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:00,315::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9d285f4-634f-4559-b4bb-384db438b79b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00465952', 'lastCheck': '6.2', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:00,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9d285f4-634f-4559-b4bb-384db438b79b`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:00,315::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:00,315::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:00,316::task::993::Storage.TaskManager.Task::(_decref) Task=`b9d285f4-634f-4559-b4bb-384db438b79b`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:42:00,337::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:00,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:00,380::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133589 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:02,403::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:02,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277806 s, 36.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:04,073::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:04,073::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:04,073::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:04,074::__init__::232::IOProcess::(_processLogs) (7536) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:04,075::__init__::232::IOProcess::(_processLogs) DEBUG|(7536) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:04,076::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:42:04,095::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421198 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:04,097::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:04,097::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:04,097::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:04,097::__init__::232::IOProcess::(_processLogs) (7537) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:04,098::__init__::232::IOProcess::(_processLogs) DEBUG|(7537) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:04,099::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:04,100::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:04,100::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:04,100::__init__::232::IOProcess::(_processLogs) (7538) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:04,101::__init__::232::IOProcess::(_processLogs) DEBUG|(7538) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:04,102::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:04,102::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:04,102::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:04,103::__init__::232::IOProcess::(_processLogs) (7539) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:04,103::__init__::232::IOProcess::(_processLogs) DEBUG|(7539) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:04,104::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:04,104::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:04,104::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:04,105::__init__::232::IOProcess::(_processLogs) (7540) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:04,106::__init__::232::IOProcess::(_processLogs) (7540) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:04,380::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:04,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:04,464::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:04,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:04,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133539 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:06,727::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:06,760::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131665 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:08,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:08,816::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131041 s, 78.1 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:42:09,426::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:42:09,427::task::595::Storage.TaskManager.Task::(_updateState) Task=`77c6508d-b810-47ef-bb1b-ce951f7560f6`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:09,463::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:09,471::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:09,473::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:09,474::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:42:09,476::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:09,476::task::1191::Storage.TaskManager.Task::(prepare) Task=`77c6508d-b810-47ef-bb1b-ce951f7560f6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:09,477::task::595::Storage.TaskManager.Task::(_updateState) Task=`77c6508d-b810-47ef-bb1b-ce951f7560f6`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:09,477::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:09,477::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:09,477::task::993::Storage.TaskManager.Task::(_decref) Task=`77c6508d-b810-47ef-bb1b-ce951f7560f6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:42:09,490::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:42:09,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`df33716f-bdeb-4676-a67c-b607502c3752`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:42:09,491::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:42:09,491::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a115931-a64a-48e6-9925-a5269fff5f88`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:42:09,492::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:42:09,492::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:42:09,492::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a115931-a64a-48e6-9925-a5269fff5f88`::Granted request >Thread-34::DEBUG::2014-08-11 15:42:09,493::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`df33716f-bdeb-4676-a67c-b607502c3752`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:42:09,493::task::993::Storage.TaskManager.Task::(_decref) Task=`df33716f-bdeb-4676-a67c-b607502c3752`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:42:09,513::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:09,513::task::1191::Storage.TaskManager.Task::(prepare) Task=`df33716f-bdeb-4676-a67c-b607502c3752`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:09,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`df33716f-bdeb-4676-a67c-b607502c3752`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:42:09,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:42:09,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:42:09,514::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:42:09,514::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:42:09,515::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:42:09,515::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:42:09,515::task::993::Storage.TaskManager.Task::(_decref) Task=`df33716f-bdeb-4676-a67c-b607502c3752`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:42:10,838::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:10,871::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133033 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:12,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:12,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132184 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:14,107::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:14,108::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:14,108::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:14,108::__init__::232::IOProcess::(_processLogs) (7541) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:14,109::__init__::232::IOProcess::(_processLogs) DEBUG|(7541) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:14,110::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:42:14,130::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414576 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:14,131::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:14,131::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:14,132::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:14,132::__init__::232::IOProcess::(_processLogs) (7542) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:14,133::__init__::232::IOProcess::(_processLogs) DEBUG|(7542) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:14,134::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:14,134::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:14,135::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:14,135::__init__::232::IOProcess::(_processLogs) (7543) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:14,136::__init__::232::IOProcess::(_processLogs) DEBUG|(7543) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:14,136::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:14,137::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:14,137::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:14,137::__init__::232::IOProcess::(_processLogs) (7544) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:14,138::__init__::232::IOProcess::(_processLogs) DEBUG|(7544) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:14,138::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:14,139::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:14,139::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:14,139::__init__::232::IOProcess::(_processLogs) (7545) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:14,140::__init__::232::IOProcess::(_processLogs) DEBUG|(7545) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:14,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:14,483::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:14,564::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:14,949::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:14,982::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131241 s, 78.0 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:42:15,547::task::595::Storage.TaskManager.Task::(_updateState) Task=`e55b4fd8-b0b2-4b3f-be38-93b93a51abfa`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:15,547::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:42:15,548::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414576', 'lastCheck': '1.4', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:15,548::task::1191::Storage.TaskManager.Task::(prepare) Task=`e55b4fd8-b0b2-4b3f-be38-93b93a51abfa`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414576', 'lastCheck': '1.4', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:15,548::task::595::Storage.TaskManager.Task::(_updateState) Task=`e55b4fd8-b0b2-4b3f-be38-93b93a51abfa`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:15,548::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:15,549::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:15,549::task::993::Storage.TaskManager.Task::(_decref) Task=`e55b4fd8-b0b2-4b3f-be38-93b93a51abfa`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:42:15,568::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:17,023::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:17,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133382 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:19,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:19,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133431 s, 76.7 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:42:19,526::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:42:19,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba4c469b-13e1-4dc7-a44d-c60c62b89273`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:19,563::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:19,571::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:19,573::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:19,574::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:42:19,576::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:19,577::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba4c469b-13e1-4dc7-a44d-c60c62b89273`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:19,577::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba4c469b-13e1-4dc7-a44d-c60c62b89273`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:19,577::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:19,577::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:19,577::task::993::Storage.TaskManager.Task::(_decref) Task=`ba4c469b-13e1-4dc7-a44d-c60c62b89273`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:42:19,588::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:42:19,588::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d0de7ba-c24d-494e-becf-2f77d60ac619`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:42:19,589::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:42:19,589::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`774285f7-f232-4f5e-8a38-3019531e307c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:42:19,590::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:42:19,590::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:42:19,590::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`774285f7-f232-4f5e-8a38-3019531e307c`::Granted request >Thread-34::DEBUG::2014-08-11 15:42:19,591::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6d0de7ba-c24d-494e-becf-2f77d60ac619`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:42:19,591::task::993::Storage.TaskManager.Task::(_decref) Task=`6d0de7ba-c24d-494e-becf-2f77d60ac619`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:42:19,611::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:19,611::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d0de7ba-c24d-494e-becf-2f77d60ac619`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:19,611::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d0de7ba-c24d-494e-becf-2f77d60ac619`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:42:19,612::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:42:19,612::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:42:19,612::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:42:19,613::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:42:19,613::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:42:19,613::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:42:19,613::task::993::Storage.TaskManager.Task::(_decref) Task=`6d0de7ba-c24d-494e-becf-2f77d60ac619`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:42:21,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:21,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134225 s, 76.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:23,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:23,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131174 s, 78.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:24,142::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:24,142::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:24,142::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:24,143::__init__::232::IOProcess::(_processLogs) (7546) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:24,143::__init__::232::IOProcess::(_processLogs) DEBUG|(7546) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:24,144::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:42:24,163::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427918 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:24,164::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:24,164::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:24,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:24,165::__init__::232::IOProcess::(_processLogs) (7547) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:24,165::__init__::232::IOProcess::(_processLogs) (7547) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:24,166::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:24,167::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:24,167::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:24,167::__init__::232::IOProcess::(_processLogs) (7548) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:24,167::__init__::232::IOProcess::(_processLogs) (7548) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:24,168::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:24,169::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:24,169::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:24,169::__init__::232::IOProcess::(_processLogs) DEBUG|(7549) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:24,169::__init__::232::IOProcess::(_processLogs) (7549) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:24,170::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:24,171::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:24,171::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:24,172::__init__::232::IOProcess::(_processLogs) (7550) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:24,172::__init__::232::IOProcess::(_processLogs) (7550) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:24,581::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:24,583::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:24,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-122::DEBUG::2014-08-11 15:42:25,154::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0da67b8-f202-4b8d-a590-524d68e67de4`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:42:25,154::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:42:25,155::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:25,156::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:25,156::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:25,156::__init__::232::IOProcess::(_processLogs) (7551) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:42:25,157::__init__::232::IOProcess::(_processLogs) DEBUG|(7551) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:25,158::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:25,159::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:42:25,159::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:42:25,160::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:42:25,160::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0da67b8-f202-4b8d-a590-524d68e67de4`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:42:25,161::__init__::232::IOProcess::(_processLogs) (7552) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:42:25,162::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0da67b8-f202-4b8d-a590-524d68e67de4`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:42:25,162::__init__::232::IOProcess::(_processLogs) (7552) Queuing response >Thread-122::DEBUG::2014-08-11 15:42:25,163::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:42:25,164::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:42:25,164::task::993::Storage.TaskManager.Task::(_decref) Task=`d0da67b8-f202-4b8d-a590-524d68e67de4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:42:25,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:25,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131624 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:27,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:27,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131957 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:29,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:29,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133035 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:29,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:29,591::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:42:29,627::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:42:29,666::task::595::Storage.TaskManager.Task::(_updateState) Task=`439ca88b-a45c-48e9-8115-4709bced129a`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:29,668::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:42:29,668::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-126::INFO::2014-08-11 15:42:29,682::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:29,683::task::1191::Storage.TaskManager.Task::(prepare) Task=`439ca88b-a45c-48e9-8115-4709bced129a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:29,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`439ca88b-a45c-48e9-8115-4709bced129a`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:29,683::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:29,683::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:29,684::task::993::Storage.TaskManager.Task::(_decref) Task=`439ca88b-a45c-48e9-8115-4709bced129a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:42:29,695::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:42:29,695::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6348839-c6ca-487e-a2d4-6cea78853a53`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:42:29,695::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:42:29,696::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`95922189-cd73-4efc-b924-6aa701084825`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:42:29,696::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:42:29,697::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:42:29,697::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`95922189-cd73-4efc-b924-6aa701084825`::Granted request >Thread-34::DEBUG::2014-08-11 15:42:29,698::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c6348839-c6ca-487e-a2d4-6cea78853a53`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:42:29,698::task::993::Storage.TaskManager.Task::(_decref) Task=`c6348839-c6ca-487e-a2d4-6cea78853a53`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:42:29,718::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:29,718::task::1191::Storage.TaskManager.Task::(prepare) Task=`c6348839-c6ca-487e-a2d4-6cea78853a53`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:29,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6348839-c6ca-487e-a2d4-6cea78853a53`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:42:29,719::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:42:29,719::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:42:29,719::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:42:29,720::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:42:29,720::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:42:29,720::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:42:29,720::task::993::Storage.TaskManager.Task::(_decref) Task=`c6348839-c6ca-487e-a2d4-6cea78853a53`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:42:30,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`5dc8649e-f586-4c92-83ed-8b10fba3b554`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:30,683::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:42:30,684::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427918', 'lastCheck': '6.5', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:30,684::task::1191::Storage.TaskManager.Task::(prepare) Task=`5dc8649e-f586-4c92-83ed-8b10fba3b554`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427918', 'lastCheck': '6.5', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:30,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`5dc8649e-f586-4c92-83ed-8b10fba3b554`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:30,684::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:30,684::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:30,685::task::993::Storage.TaskManager.Task::(_decref) Task=`5dc8649e-f586-4c92-83ed-8b10fba3b554`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:42:30,712::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:31,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:31,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132547 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:33,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:33,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133279 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:34,173::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:34,174::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:34,174::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:34,174::__init__::232::IOProcess::(_processLogs) (7553) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:34,174::__init__::232::IOProcess::(_processLogs) (7553) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:34,175::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:42:34,195::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422506 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:34,196::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:34,196::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:34,197::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:34,197::__init__::232::IOProcess::(_processLogs) (7554) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:34,197::__init__::232::IOProcess::(_processLogs) (7554) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:34,198::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:34,199::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:34,199::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:34,199::__init__::232::IOProcess::(_processLogs) (7555) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:34,200::__init__::232::IOProcess::(_processLogs) DEBUG|(7555) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:34,200::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:34,201::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:34,202::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:34,202::__init__::232::IOProcess::(_processLogs) (7556) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:34,202::__init__::232::IOProcess::(_processLogs) (7556) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:34,202::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:34,203::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:34,203::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:34,203::__init__::232::IOProcess::(_processLogs) (7557) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:34,204::__init__::232::IOProcess::(_processLogs) DEBUG|(7557) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:34,677::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:34,678::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:34,679::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:35,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:35,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141903 s, 72.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:37,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:37,618::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132996 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:39,640::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:39,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131192 s, 78.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:39,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:39,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::DEBUG::2014-08-11 15:42:39,733::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:42:39,766::task::595::Storage.TaskManager.Task::(_updateState) Task=`566f2f31-a236-43b9-854c-970d116d6118`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:39,768::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:39,768::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:42:39,782::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:39,783::task::1191::Storage.TaskManager.Task::(prepare) Task=`566f2f31-a236-43b9-854c-970d116d6118`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:39,783::task::595::Storage.TaskManager.Task::(_updateState) Task=`566f2f31-a236-43b9-854c-970d116d6118`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:39,783::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:39,784::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:39,784::task::993::Storage.TaskManager.Task::(_decref) Task=`566f2f31-a236-43b9-854c-970d116d6118`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:42:39,798::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:42:39,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5e4368e-0b1c-4229-a80e-9beabd0865d6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:42:39,802::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:42:39,804::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1fd2106e-05bf-44d2-8138-763f5adc1845`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:42:39,804::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:42:39,805::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:42:39,805::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1fd2106e-05bf-44d2-8138-763f5adc1845`::Granted request >Thread-34::DEBUG::2014-08-11 15:42:39,805::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b5e4368e-0b1c-4229-a80e-9beabd0865d6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:42:39,806::task::993::Storage.TaskManager.Task::(_decref) Task=`b5e4368e-0b1c-4229-a80e-9beabd0865d6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:42:39,825::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:39,826::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5e4368e-0b1c-4229-a80e-9beabd0865d6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:39,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5e4368e-0b1c-4229-a80e-9beabd0865d6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:42:39,826::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:42:39,826::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:42:39,827::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:42:39,827::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:42:39,827::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:42:39,827::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:42:39,828::task::993::Storage.TaskManager.Task::(_decref) Task=`b5e4368e-0b1c-4229-a80e-9beabd0865d6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:42:41,703::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:41,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131566 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:43,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:43,791::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132648 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:44,205::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:44,206::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:44,206::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:44,206::__init__::232::IOProcess::(_processLogs) (7558) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:44,207::__init__::232::IOProcess::(_processLogs) DEBUG|(7558) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:44,208::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:42:44,227::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427355 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:44,228::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:44,229::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:44,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:44,229::__init__::232::IOProcess::(_processLogs) (7559) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:44,230::__init__::232::IOProcess::(_processLogs) DEBUG|(7559) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:44,231::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:44,232::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:44,232::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:44,233::__init__::232::IOProcess::(_processLogs) (7560) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:44,233::__init__::232::IOProcess::(_processLogs) (7560) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:44,233::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:44,234::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:44,234::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:44,235::__init__::232::IOProcess::(_processLogs) DEBUG|(7561) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:44,235::__init__::232::IOProcess::(_processLogs) (7561) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:44,236::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:44,236::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:44,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:44,238::__init__::232::IOProcess::(_processLogs) (7562) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:44,238::__init__::232::IOProcess::(_processLogs) (7562) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:44,774::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:44,864::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:44,865::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:45,814::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-126::DEBUG::2014-08-11 15:42:45,835::task::595::Storage.TaskManager.Task::(_updateState) Task=`35ee6dd4-d56a-4c0d-b5e6-2556796f7db4`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:45,837::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:42:45,838::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427355', 'lastCheck': '1.6', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:45,839::task::1191::Storage.TaskManager.Task::(prepare) Task=`35ee6dd4-d56a-4c0d-b5e6-2556796f7db4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427355', 'lastCheck': '1.6', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:42:45,840::task::595::Storage.TaskManager.Task::(_updateState) Task=`35ee6dd4-d56a-4c0d-b5e6-2556796f7db4`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:45,840::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:45,840::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:45,841::task::993::Storage.TaskManager.Task::(_decref) Task=`35ee6dd4-d56a-4c0d-b5e6-2556796f7db4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:42:45,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0228818 s, 44.8 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:42:45,872::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:47,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:47,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132057 s, 77.5 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:42:49,841::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:42:49,864::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c36106b-3c0e-4139-8ff9-7afd99b200c1`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:49,864::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:49,872::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:49,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:49,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:42:49,877::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:49,877::task::1191::Storage.TaskManager.Task::(prepare) Task=`1c36106b-3c0e-4139-8ff9-7afd99b200c1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:49,878::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c36106b-3c0e-4139-8ff9-7afd99b200c1`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:49,878::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:49,878::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:49,878::task::993::Storage.TaskManager.Task::(_decref) Task=`1c36106b-3c0e-4139-8ff9-7afd99b200c1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:42:49,888::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:42:49,889::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc07171e-d647-428e-a4fb-41c34fa4bd9a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:42:49,889::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:42:49,890::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4f52f945-6b3d-4fe0-81db-aa2f83f3226d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:42:49,890::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:42:49,891::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:42:49,891::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4f52f945-6b3d-4fe0-81db-aa2f83f3226d`::Granted request >Thread-34::DEBUG::2014-08-11 15:42:49,893::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dc07171e-d647-428e-a4fb-41c34fa4bd9a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:42:49,894::task::993::Storage.TaskManager.Task::(_decref) Task=`dc07171e-d647-428e-a4fb-41c34fa4bd9a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:42:49,913::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:49,914::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc07171e-d647-428e-a4fb-41c34fa4bd9a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:42:49,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc07171e-d647-428e-a4fb-41c34fa4bd9a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:42:49,914::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:42:49,914::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:42:49,915::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:42:49,915::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:42:49,915::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:42:49,915::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:42:49,916::task::993::Storage.TaskManager.Task::(_decref) Task=`dc07171e-d647-428e-a4fb-41c34fa4bd9a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:42:49,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:49,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132623 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:52,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:52,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132143 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:54,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:54,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132977 s, 77.0 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:42:54,238::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:42:54,240::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,240::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,241::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,241::__init__::232::IOProcess::(_processLogs) (7563) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:42:54,242::__init__::232::IOProcess::(_processLogs) DEBUG|(7563) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:54,252::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:42:54,252::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,253::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,253::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,253::__init__::232::IOProcess::(_processLogs) (7564) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:42:54,254::__init__::232::IOProcess::(_processLogs) DEBUG|(7564) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:54,255::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:42:54,256::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,256::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,256::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,256::__init__::232::IOProcess::(_processLogs) (7565) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:42:54,257::__init__::232::IOProcess::(_processLogs) DEBUG|(7565) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:54,258::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:42:54,259::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,259::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,260::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,260::__init__::232::IOProcess::(_processLogs) (7566) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:54,261::__init__::232::IOProcess::(_processLogs) (7566) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:54,261::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,261::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,262::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,262::__init__::232::IOProcess::(_processLogs) (7567) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:42:54,262::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 15:42:54,263::__init__::232::IOProcess::(_processLogs) DEBUG|(7567) Queuing response >Thread-16::INFO::2014-08-11 15:42:54,264::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:42:54,264::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:42:54,265::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,265::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,266::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,266::__init__::232::IOProcess::(_processLogs) (7568) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:54,267::__init__::232::IOProcess::(_processLogs) DEBUG|(7568) Queuing response >Thread-16::DEBUG::2014-08-11 15:42:54,268::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:42:54,287::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418018 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:42:54,289::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,289::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,289::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,290::__init__::232::IOProcess::(_processLogs) (7569) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:42:54,290::__init__::232::IOProcess::(_processLogs) DEBUG|(7569) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:54,291::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,292::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,292::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,293::__init__::232::IOProcess::(_processLogs) (7570) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:54,293::__init__::232::IOProcess::(_processLogs) (7570) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:54,293::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,294::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,294::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,295::__init__::232::IOProcess::(_processLogs) (7571) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:54,295::__init__::232::IOProcess::(_processLogs) (7571) Queuing response >Thread-14::DEBUG::2014-08-11 15:42:54,295::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:42:54,296::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:42:54,297::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:42:54,297::__init__::232::IOProcess::(_processLogs) (7572) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:42:54,298::__init__::232::IOProcess::(_processLogs) (7572) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:54,882::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:54,883::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:54,964::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:42:56,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:56,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133323 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:42:58,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:42:58,207::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013184 s, 77.7 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:42:59,928::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-126::DEBUG::2014-08-11 15:42:59,928::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f9f6027-0347-47fd-8a77-ad32eed5c50e`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:42:59,963::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:59,971::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:59,973::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:42:59,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:42:59,976::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:59,977::task::1191::Storage.TaskManager.Task::(prepare) Task=`0f9f6027-0347-47fd-8a77-ad32eed5c50e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:42:59,977::task::595::Storage.TaskManager.Task::(_updateState) Task=`0f9f6027-0347-47fd-8a77-ad32eed5c50e`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:42:59,977::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:42:59,977::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:42:59,977::task::993::Storage.TaskManager.Task::(_decref) Task=`0f9f6027-0347-47fd-8a77-ad32eed5c50e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:42:59,988::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-34::DEBUG::2014-08-11 15:42:59,989::task::595::Storage.TaskManager.Task::(_updateState) Task=`1215ee74-c2ca-4d7c-b42d-c7da51ea3c14`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:42:59,989::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:42:59,990::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`43aa3c54-ac97-4028-b969-97ef14416d83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:42:59,990::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:42:59,991::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:42:59,991::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`43aa3c54-ac97-4028-b969-97ef14416d83`::Granted request >Thread-34::DEBUG::2014-08-11 15:42:59,991::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1215ee74-c2ca-4d7c-b42d-c7da51ea3c14`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:42:59,992::task::993::Storage.TaskManager.Task::(_decref) Task=`1215ee74-c2ca-4d7c-b42d-c7da51ea3c14`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:43:00,012::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:00,012::task::1191::Storage.TaskManager.Task::(prepare) Task=`1215ee74-c2ca-4d7c-b42d-c7da51ea3c14`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:00,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`1215ee74-c2ca-4d7c-b42d-c7da51ea3c14`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:43:00,013::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:43:00,013::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:43:00,013::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:43:00,013::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:43:00,014::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:43:00,014::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:43:00,014::task::993::Storage.TaskManager.Task::(_decref) Task=`1215ee74-c2ca-4d7c-b42d-c7da51ea3c14`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:00,230::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:00,262::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132804 s, 77.1 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:43:00,983::task::595::Storage.TaskManager.Task::(_updateState) Task=`821086e0-2f6b-4632-b643-2fd72430ce4b`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:43:00,983::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-126::INFO::2014-08-11 15:43:00,983::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418018', 'lastCheck': '6.7', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:43:00,984::task::1191::Storage.TaskManager.Task::(prepare) Task=`821086e0-2f6b-4632-b643-2fd72430ce4b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418018', 'lastCheck': '6.7', 'valid': True}} >Thread-126::DEBUG::2014-08-11 15:43:00,984::task::595::Storage.TaskManager.Task::(_updateState) Task=`821086e0-2f6b-4632-b643-2fd72430ce4b`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:43:00,984::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:43:00,984::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:43:00,985::task::993::Storage.TaskManager.Task::(_decref) Task=`821086e0-2f6b-4632-b643-2fd72430ce4b`::ref 0 aborting False >Thread-126::DEBUG::2014-08-11 15:43:01,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:02,287::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:02,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250666 s, 40.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:04,298::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:04,299::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:04,300::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:43:04,300::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:43:04,301::__init__::232::IOProcess::(_processLogs) (7573) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:04,320::__init__::232::IOProcess::(_processLogs) (7573) Queuing response >Thread-16::DEBUG::2014-08-11 15:43:04,321::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000760279 s, 598 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:04,322::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:04,322::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:04,323::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:04,323::__init__::232::IOProcess::(_processLogs) (7574) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:04,324::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:04,324::__init__::232::IOProcess::(_processLogs) (7575) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:04,325::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:04,325::__init__::232::IOProcess::(_processLogs) (7576) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:04,326::__init__::232::IOProcess::(_processLogs) (7576) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:04,327::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:04,328::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:04,328::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:04,329::__init__::232::IOProcess::(_processLogs) (7577) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:04,329::__init__::232::IOProcess::(_processLogs) (7577) Queuing response >Dummy-33::DEBUG::2014-08-11 15:43:04,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:04,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132128 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:04,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:04,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:05,064::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:06,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:06,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132027 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:08,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:08,620::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132964 s, 77.0 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-08-11 15:43:10,038::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-126::DEBUG::2014-08-11 15:43:10,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`371af961-9cc8-48f0-9474-5849acead674`::moving from state init -> state preparing >Thread-126::INFO::2014-08-11 15:43:10,063::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:10,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:10,073::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:10,074::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-126::INFO::2014-08-11 15:43:10,076::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:43:10,077::task::1191::Storage.TaskManager.Task::(prepare) Task=`371af961-9cc8-48f0-9474-5849acead674`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-126::DEBUG::2014-08-11 15:43:10,077::task::595::Storage.TaskManager.Task::(_updateState) Task=`371af961-9cc8-48f0-9474-5849acead674`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-08-11 15:43:10,078::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-08-11 15:43:10,078::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-08-11 15:43:10,078::task::993::Storage.TaskManager.Task::(_decref) Task=`371af961-9cc8-48f0-9474-5849acead674`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 15:43:10,079::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:52027 >Detector thread::DEBUG::2014-08-11 15:43:10,080::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:52027 >Detector thread::DEBUG::2014-08-11 15:43:10,080::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:52027 >Detector thread::DEBUG::2014-08-11 15:43:10,080::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 52027) >Thread-34::DEBUG::2014-08-11 15:43:10,093::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:43:10,093::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ef9ff2c-d92d-472a-9ec4-862879932779`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:43:10,094::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:43:10,095::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f3cf1d8b-1c6a-4ca6-99de-3b7bbb0da602`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:43:10,095::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:43:10,095::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:43:10,095::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f3cf1d8b-1c6a-4ca6-99de-3b7bbb0da602`::Granted request >Thread-34::DEBUG::2014-08-11 15:43:10,096::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1ef9ff2c-d92d-472a-9ec4-862879932779`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:43:10,096::task::993::Storage.TaskManager.Task::(_decref) Task=`1ef9ff2c-d92d-472a-9ec4-862879932779`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:43:10,116::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:10,116::task::1191::Storage.TaskManager.Task::(prepare) Task=`1ef9ff2c-d92d-472a-9ec4-862879932779`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:10,117::task::595::Storage.TaskManager.Task::(_updateState) Task=`1ef9ff2c-d92d-472a-9ec4-862879932779`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:43:10,117::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:43:10,117::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:43:10,118::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:43:10,118::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:43:10,118::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:43:10,118::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:43:10,119::task::993::Storage.TaskManager.Task::(_decref) Task=`1ef9ff2c-d92d-472a-9ec4-862879932779`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:10,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:10,676::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136861 s, 74.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:12,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:12,731::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131771 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:14,330::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:14,330::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:14,331::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:14,331::__init__::232::IOProcess::(_processLogs) (7578) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:14,331::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|(7578) Queuing response >Thread-16::DEBUG::2014-08-11 15:43:14,332::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:43:14,350::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423861 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:14,352::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:14,352::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:14,352::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:14,352::__init__::232::IOProcess::(_processLogs) (7579) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:14,353::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|(7579) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:14,354::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:14,354::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:14,354::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:14,355::__init__::232::IOProcess::(_processLogs) (7580) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:14,355::__init__::232::IOProcess::(_processLogs) (7580) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:14,356::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:14,356::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:14,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:14,357::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|(7581) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:14,357::__init__::232::IOProcess::(_processLogs) (7581) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:14,358::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:14,358::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:14,358::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:14,358::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|(7582) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:14,359::__init__::232::IOProcess::(_processLogs) (7582) Queuing response >Dummy-33::DEBUG::2014-08-11 15:43:14,754::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:14,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131738 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:15,082::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:15,083::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:15,164::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:43:16,119::task::595::Storage.TaskManager.Task::(_updateState) Task=`82ad109d-f2c5-4522-9a2d-013a93ba9627`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:43:16,120::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:43:16,120::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423861', 'lastCheck': '1.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:43:16,120::task::1191::Storage.TaskManager.Task::(prepare) Task=`82ad109d-f2c5-4522-9a2d-013a93ba9627`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423861', 'lastCheck': '1.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:43:16,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`82ad109d-f2c5-4522-9a2d-013a93ba9627`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:43:16,121::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:43:16,121::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:43:16,121::task::993::Storage.TaskManager.Task::(_decref) Task=`82ad109d-f2c5-4522-9a2d-013a93ba9627`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:43:16,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:16,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:16,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013351 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:18,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:18,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132669 s, 77.2 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:43:20,136::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-128::DEBUG::2014-08-11 15:43:20,137::task::595::Storage.TaskManager.Task::(_updateState) Task=`1fb8bb9f-9e90-45b7-b236-bed34c8b2cab`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:43:20,163::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:20,170::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:20,173::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:20,174::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:43:20,176::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:20,177::task::1191::Storage.TaskManager.Task::(prepare) Task=`1fb8bb9f-9e90-45b7-b236-bed34c8b2cab`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:20,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`1fb8bb9f-9e90-45b7-b236-bed34c8b2cab`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:43:20,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:43:20,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:43:20,177::task::993::Storage.TaskManager.Task::(_decref) Task=`1fb8bb9f-9e90-45b7-b236-bed34c8b2cab`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:43:20,187::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 15:43:20,188::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6127278-abec-4b26-b821-77c06f463558`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:43:20,188::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:43:20,189::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`31adffe1-fe0f-45ce-bfe8-a9f6f4515d17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:43:20,189::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:43:20,190::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:43:20,190::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`31adffe1-fe0f-45ce-bfe8-a9f6f4515d17`::Granted request >Thread-34::DEBUG::2014-08-11 15:43:20,190::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c6127278-abec-4b26-b821-77c06f463558`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:43:20,191::task::993::Storage.TaskManager.Task::(_decref) Task=`c6127278-abec-4b26-b821-77c06f463558`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:43:20,210::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:20,211::task::1191::Storage.TaskManager.Task::(prepare) Task=`c6127278-abec-4b26-b821-77c06f463558`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952150016', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:20,211::task::595::Storage.TaskManager.Task::(_updateState) Task=`c6127278-abec-4b26-b821-77c06f463558`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:43:20,211::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:43:20,212::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:43:20,212::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:43:20,212::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:43:20,212::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:43:20,213::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:43:20,213::task::993::Storage.TaskManager.Task::(_decref) Task=`c6127278-abec-4b26-b821-77c06f463558`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:20,921::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:20,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132059 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:22,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:23,010::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131853 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:24,361::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:24,361::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:24,361::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:24,362::__init__::232::IOProcess::(_processLogs) (7583) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:24,362::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|(7583) Queuing response >Thread-16::DEBUG::2014-08-11 15:43:24,363::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:43:24,381::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418281 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:24,383::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:24,383::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:24,383::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:24,384::__init__::232::IOProcess::(_processLogs) (7584) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:24,384::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|(7584) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:24,385::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:24,385::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:24,386::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:24,386::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|(7585) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:24,386::__init__::232::IOProcess::(_processLogs) (7585) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:24,387::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:24,387::__init__::232::IOProcess::(_processLogs) (7576) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:24,387::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:24,388::__init__::232::IOProcess::(_processLogs) (7586) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:24,388::__init__::232::IOProcess::(_processLogs) (7586) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:24,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:24,389::__init__::232::IOProcess::(_processLogs) (7587) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:24,389::__init__::232::IOProcess::(_processLogs) (7587) Queuing response >Dummy-33::DEBUG::2014-08-11 15:43:25,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:25,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134342 s, 76.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:25,181::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:25,182::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-122::DEBUG::2014-08-11 15:43:25,266::task::595::Storage.TaskManager.Task::(_updateState) Task=`0cdfcc97-06c7-4240-a8a7-53c308b18cb3`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:43:25,268::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:25,270::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:43:25,271::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:25,273::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:25,274::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:25,275::__init__::232::IOProcess::(_processLogs) (7588) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:43:25,275::__init__::232::IOProcess::(_processLogs) DEBUG|(7588) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:25,276::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:25,276::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:25,276::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:25,277::__init__::232::IOProcess::(_processLogs) (7589) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:43:25,277::__init__::232::IOProcess::(_processLogs) (7589) Queuing response >Thread-122::INFO::2014-08-11 15:43:25,277::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:43:25,278::task::1191::Storage.TaskManager.Task::(prepare) Task=`0cdfcc97-06c7-4240-a8a7-53c308b18cb3`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:43:25,278::task::595::Storage.TaskManager.Task::(_updateState) Task=`0cdfcc97-06c7-4240-a8a7-53c308b18cb3`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:43:25,278::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:43:25,278::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:43:25,279::task::993::Storage.TaskManager.Task::(_decref) Task=`0cdfcc97-06c7-4240-a8a7-53c308b18cb3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:27,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:27,122::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133437 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:29,145::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:29,180::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155315 s, 65.9 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:43:30,226::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:43:30,227::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00b83d3-63a0-4399-8f16-b5315e309973`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:43:30,228::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:43:30,241::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:30,241::task::1191::Storage.TaskManager.Task::(prepare) Task=`f00b83d3-63a0-4399-8f16-b5315e309973`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:30,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`f00b83d3-63a0-4399-8f16-b5315e309973`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:43:30,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:43:30,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:43:30,242::task::993::Storage.TaskManager.Task::(_decref) Task=`f00b83d3-63a0-4399-8f16-b5315e309973`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:43:30,254::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:43:30,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b5594c9-ef6c-4e5a-b8b6-b34f98e821ac`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:43:30,255::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:43:30,255::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ca28d3af-e9d9-4c7f-99d1-f9ee2c25058e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:43:30,256::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:43:30,256::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:43:30,256::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ca28d3af-e9d9-4c7f-99d1-f9ee2c25058e`::Granted request >Thread-34::DEBUG::2014-08-11 15:43:30,257::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0b5594c9-ef6c-4e5a-b8b6-b34f98e821ac`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:43:30,257::task::993::Storage.TaskManager.Task::(_decref) Task=`0b5594c9-ef6c-4e5a-b8b6-b34f98e821ac`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:43:30,277::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:30,279::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b5594c9-ef6c-4e5a-b8b6-b34f98e821ac`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:30,280::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:43:30,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b5594c9-ef6c-4e5a-b8b6-b34f98e821ac`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:43:30,282::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:43:30,283::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:30,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:43:30,286::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:43:30,288::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:43:30,289::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:30,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:43:30,290::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:43:30,293::task::993::Storage.TaskManager.Task::(_decref) Task=`0b5594c9-ef6c-4e5a-b8b6-b34f98e821ac`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:31,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-128::DEBUG::2014-08-11 15:43:31,230::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f9442f6-fe89-40be-a7be-4b266e7cd6cc`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:43:31,231::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:43:31,231::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418281', 'lastCheck': '6.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:43:31,231::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f9442f6-fe89-40be-a7be-4b266e7cd6cc`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418281', 'lastCheck': '6.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:43:31,232::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f9442f6-fe89-40be-a7be-4b266e7cd6cc`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:43:31,232::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:43:31,232::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:43:31,232::task::993::Storage.TaskManager.Task::(_decref) Task=`3f9442f6-fe89-40be-a7be-4b266e7cd6cc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:31,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0290243 s, 35.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:43:31,268::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:33,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:33,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134032 s, 76.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:34,391::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:34,392::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:34,392::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:34,392::__init__::232::IOProcess::(_processLogs) (7590) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:34,392::__init__::232::IOProcess::(_processLogs) DEBUG|(7590) Queuing response >Thread-16::DEBUG::2014-08-11 15:43:34,393::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:43:34,412::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420969 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:34,414::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:34,414::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:34,414::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:34,415::__init__::232::IOProcess::(_processLogs) (7591) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:34,415::__init__::232::IOProcess::(_processLogs) DEBUG|(7591) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:34,417::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:34,417::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:34,417::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:34,418::__init__::232::IOProcess::(_processLogs) (7592) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:34,418::__init__::232::IOProcess::(_processLogs) DEBUG|(7592) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:34,419::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:34,419::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:34,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:34,420::__init__::232::IOProcess::(_processLogs) (7593) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:34,420::__init__::232::IOProcess::(_processLogs) (7593) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:34,421::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:34,421::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:34,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:34,422::__init__::232::IOProcess::(_processLogs) (7594) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:34,422::__init__::232::IOProcess::(_processLogs) (7594) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:35,299::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:35,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:35,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:35,388::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:35,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0203906 s, 50.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:37,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:37,465::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013588 s, 75.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:39,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:39,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131991 s, 77.6 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:43:40,307::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:43:40,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`e141d927-0bad-41e0-957f-b52bea6f8e35`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:43:40,365::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:43:40,382::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:40,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`e141d927-0bad-41e0-957f-b52bea6f8e35`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:40,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`e141d927-0bad-41e0-957f-b52bea6f8e35`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:43:40,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:43:40,383::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:43:40,383::task::993::Storage.TaskManager.Task::(_decref) Task=`e141d927-0bad-41e0-957f-b52bea6f8e35`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:40,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:40,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:40,403::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:43:40,405::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:43:40,405::task::595::Storage.TaskManager.Task::(_updateState) Task=`78a9d467-615d-4035-9947-e603abe40c3b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:43:40,406::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:43:40,406::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eb91f5d5-8fd1-4d9e-a3c7-818c6efeb8d4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:43:40,406::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:43:40,407::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:43:40,407::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eb91f5d5-8fd1-4d9e-a3c7-818c6efeb8d4`::Granted request >Thread-34::DEBUG::2014-08-11 15:43:40,408::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`78a9d467-615d-4035-9947-e603abe40c3b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:43:40,408::task::993::Storage.TaskManager.Task::(_decref) Task=`78a9d467-615d-4035-9947-e603abe40c3b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:43:40,428::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:40,429::task::1191::Storage.TaskManager.Task::(prepare) Task=`78a9d467-615d-4035-9947-e603abe40c3b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:40,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`78a9d467-615d-4035-9947-e603abe40c3b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:43:40,429::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:43:40,429::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:43:40,430::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:43:40,430::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:43:40,430::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:43:40,430::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:43:40,431::task::993::Storage.TaskManager.Task::(_decref) Task=`78a9d467-615d-4035-9947-e603abe40c3b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:41,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:41,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131199 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:43,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:43,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01325 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:44,424::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:44,425::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:44,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:43:44,426::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:43:44,427::__init__::232::IOProcess::(_processLogs) (7595) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:44,446::__init__::232::IOProcess::(_processLogs) (7595) Queuing response >Thread-16::DEBUG::2014-08-11 15:43:44,447::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000752141 s, 605 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:44,448::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:43:44,448::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:43:44,449::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:44,449::__init__::232::IOProcess::(_processLogs) (7596) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:44,450::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:44,450::__init__::232::IOProcess::(_processLogs) (7597) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:44,451::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:44,452::__init__::232::IOProcess::(_processLogs) (7598) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:44,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:44,453::__init__::232::IOProcess::(_processLogs) (7599) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:45,410::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:45,411::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:45,466::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:45,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:45,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131762 s, 77.7 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:43:46,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b6f3fd5-cfb1-4428-9add-b97b3fd47f57`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:43:46,382::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:43:46,383::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000752141', 'lastCheck': '1.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:43:46,383::task::1191::Storage.TaskManager.Task::(prepare) Task=`9b6f3fd5-cfb1-4428-9add-b97b3fd47f57`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000752141', 'lastCheck': '1.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:43:46,383::task::595::Storage.TaskManager.Task::(_updateState) Task=`9b6f3fd5-cfb1-4428-9add-b97b3fd47f57`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:43:46,383::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:43:46,383::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:43:46,384::task::993::Storage.TaskManager.Task::(_decref) Task=`9b6f3fd5-cfb1-4428-9add-b97b3fd47f57`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:43:46,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:47,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:47,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132037 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:49,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:49,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133145 s, 76.9 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:43:50,444::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:43:50,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`d903f2b6-7c9b-4c70-a679-fd548f9b778d`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:43:50,466::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:50,474::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:50,475::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:50,476::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:43:50,479::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:50,480::task::1191::Storage.TaskManager.Task::(prepare) Task=`d903f2b6-7c9b-4c70-a679-fd548f9b778d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:43:50,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`d903f2b6-7c9b-4c70-a679-fd548f9b778d`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:43:50,480::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:43:50,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:43:50,480::task::993::Storage.TaskManager.Task::(_decref) Task=`d903f2b6-7c9b-4c70-a679-fd548f9b778d`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:43:50,493::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:43:50,494::task::595::Storage.TaskManager.Task::(_updateState) Task=`b08c6f8c-adba-4254-b72e-6fb58fae3a58`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:43:50,494::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:43:50,495::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9509744c-54dc-422e-9148-6d12ab45f66d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:43:50,495::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:43:50,495::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:43:50,496::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9509744c-54dc-422e-9148-6d12ab45f66d`::Granted request >Thread-34::DEBUG::2014-08-11 15:43:50,496::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b08c6f8c-adba-4254-b72e-6fb58fae3a58`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:43:50,496::task::993::Storage.TaskManager.Task::(_decref) Task=`b08c6f8c-adba-4254-b72e-6fb58fae3a58`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:43:50,516::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:50,517::task::1191::Storage.TaskManager.Task::(prepare) Task=`b08c6f8c-adba-4254-b72e-6fb58fae3a58`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:43:50,517::task::595::Storage.TaskManager.Task::(_updateState) Task=`b08c6f8c-adba-4254-b72e-6fb58fae3a58`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:43:50,517::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:43:50,517::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:43:50,518::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:43:50,518::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:43:50,518::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:43:50,519::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:43:50,519::task::993::Storage.TaskManager.Task::(_decref) Task=`b08c6f8c-adba-4254-b72e-6fb58fae3a58`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:43:51,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:51,854::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133482 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:53,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:53,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131692 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:54,455::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:54,455::__init__::232::IOProcess::(_processLogs) (7600) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:43:54,456::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:43:54,476::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424193 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:43:54,477::__init__::232::IOProcess::(_processLogs) (7601) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:43:54,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:54,479::__init__::232::IOProcess::(_processLogs) (7602) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:54,479::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:54,480::__init__::232::IOProcess::(_processLogs) (7603) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:54,480::__init__::232::IOProcess::(_processLogs) (7603) Queuing response >Thread-14::DEBUG::2014-08-11 15:43:54,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:43:54,481::__init__::232::IOProcess::(_processLogs) (7604) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:43:54,482::__init__::232::IOProcess::(_processLogs) (7604) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:55,485::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:55,489::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:43:55,567::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:43:55,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:55,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132532 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:43:57,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:43:58,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132616 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:00,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:00,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134479 s, 76.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:44:00,531::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-128::DEBUG::2014-08-11 15:44:00,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ed754c4-c1f4-4a6d-b684-b46df0be94f5`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:00,566::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:00,575::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:00,576::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:00,577::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:44:00,579::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:00,579::task::1191::Storage.TaskManager.Task::(prepare) Task=`3ed754c4-c1f4-4a6d-b684-b46df0be94f5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:00,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`3ed754c4-c1f4-4a6d-b684-b46df0be94f5`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:00,580::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:00,580::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:00,580::task::993::Storage.TaskManager.Task::(_decref) Task=`3ed754c4-c1f4-4a6d-b684-b46df0be94f5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:44:00,593::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 15:44:00,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`378361b0-2ae7-4063-8e44-9580275b0df8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:44:00,594::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:44:00,594::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`002c25e1-9a6f-42b9-82a3-04c2540f0714`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:44:00,594::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:44:00,595::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:44:00,595::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`002c25e1-9a6f-42b9-82a3-04c2540f0714`::Granted request >Thread-34::DEBUG::2014-08-11 15:44:00,596::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`378361b0-2ae7-4063-8e44-9580275b0df8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:44:00,596::task::993::Storage.TaskManager.Task::(_decref) Task=`378361b0-2ae7-4063-8e44-9580275b0df8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:44:00,616::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:00,616::task::1191::Storage.TaskManager.Task::(prepare) Task=`378361b0-2ae7-4063-8e44-9580275b0df8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:00,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`378361b0-2ae7-4063-8e44-9580275b0df8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:44:00,617::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:44:00,617::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:44:00,617::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:44:00,618::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:44:00,618::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:44:00,618::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:44:00,618::task::993::Storage.TaskManager.Task::(_decref) Task=`378361b0-2ae7-4063-8e44-9580275b0df8`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:44:01,499::task::595::Storage.TaskManager.Task::(_updateState) Task=`72adac9e-6d12-408e-871f-747a450c2dc5`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:01,499::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:44:01,499::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424193', 'lastCheck': '7.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:01,500::task::1191::Storage.TaskManager.Task::(prepare) Task=`72adac9e-6d12-408e-871f-747a450c2dc5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424193', 'lastCheck': '7.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:01,500::task::595::Storage.TaskManager.Task::(_updateState) Task=`72adac9e-6d12-408e-871f-747a450c2dc5`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:01,500::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:01,500::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:01,500::task::993::Storage.TaskManager.Task::(_decref) Task=`72adac9e-6d12-408e-871f-747a450c2dc5`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:44:01,525::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:02,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:02,150::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276083 s, 37.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:04,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:04,378::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132759 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:04,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:04,484::__init__::232::IOProcess::(_processLogs) (7605) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:44:04,485::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:44:04,504::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042299 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:04,506::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:04,506::__init__::232::IOProcess::(_processLogs) (7606) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:04,507::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:04,508::__init__::232::IOProcess::(_processLogs) (7607) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:04,509::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:04,509::__init__::232::IOProcess::(_processLogs) (7608) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:04,509::__init__::232::IOProcess::(_processLogs) (7608) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:04,510::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:04,510::__init__::232::IOProcess::(_processLogs) (7609) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:04,510::__init__::232::IOProcess::(_processLogs) (7609) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:05,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:05,586::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:05,667::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:06,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:06,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013126 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:08,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:08,489::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131909 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:10,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:10,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132662 s, 77.2 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:44:10,629::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:44:10,629::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fad62de-826d-4f54-9456-770e009da1b7`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:10,666::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:10,674::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:10,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:10,677::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:44:10,679::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:10,680::task::1191::Storage.TaskManager.Task::(prepare) Task=`6fad62de-826d-4f54-9456-770e009da1b7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:10,680::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fad62de-826d-4f54-9456-770e009da1b7`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:10,680::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:10,680::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:10,680::task::993::Storage.TaskManager.Task::(_decref) Task=`6fad62de-826d-4f54-9456-770e009da1b7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:44:10,692::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:44:10,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ccc8302-9665-49cb-ad86-fb02df6642b9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:44:10,693::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:44:10,694::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cf42e41c-c765-491e-b53b-05699ee5be8c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:44:10,694::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:44:10,694::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:44:10,695::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`cf42e41c-c765-491e-b53b-05699ee5be8c`::Granted request >Thread-34::DEBUG::2014-08-11 15:44:10,695::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6ccc8302-9665-49cb-ad86-fb02df6642b9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:44:10,695::task::993::Storage.TaskManager.Task::(_decref) Task=`6ccc8302-9665-49cb-ad86-fb02df6642b9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:44:10,715::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:10,715::task::1191::Storage.TaskManager.Task::(prepare) Task=`6ccc8302-9665-49cb-ad86-fb02df6642b9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:10,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`6ccc8302-9665-49cb-ad86-fb02df6642b9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:44:10,716::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:44:10,716::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:44:10,716::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:44:10,717::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:44:10,717::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:44:10,717::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:44:10,718::task::993::Storage.TaskManager.Task::(_decref) Task=`6ccc8302-9665-49cb-ad86-fb02df6642b9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:44:12,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:12,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132392 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:14,512::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:14,513::__init__::232::IOProcess::(_processLogs) (7610) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:44:14,514::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:44:14,533::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041741 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:14,535::__init__::232::IOProcess::(_processLogs) (7611) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:14,536::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:14,536::__init__::232::IOProcess::(_processLogs) (7612) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:14,537::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:14,537::__init__::232::IOProcess::(_processLogs) (7613) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:14,538::__init__::232::IOProcess::(_processLogs) (7613) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:14,539::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:14,539::__init__::232::IOProcess::(_processLogs) (7614) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:14,539::__init__::232::IOProcess::(_processLogs) (7614) Queuing response >Dummy-33::DEBUG::2014-08-11 15:44:14,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:14,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131762 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:15,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:15,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:15,768::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:44:16,644::task::595::Storage.TaskManager.Task::(_updateState) Task=`83897800-7a89-4ffc-951a-c455276a92d6`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:16,644::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:44:16,644::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041741', 'lastCheck': '2.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:16,645::task::1191::Storage.TaskManager.Task::(prepare) Task=`83897800-7a89-4ffc-951a-c455276a92d6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041741', 'lastCheck': '2.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:16,645::task::595::Storage.TaskManager.Task::(_updateState) Task=`83897800-7a89-4ffc-951a-c455276a92d6`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:16,645::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:16,645::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:16,646::task::993::Storage.TaskManager.Task::(_decref) Task=`83897800-7a89-4ffc-951a-c455276a92d6`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:44:16,666::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:16,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:16,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132458 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:18,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:18,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134562 s, 76.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:44:20,730::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:44:20,731::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a2ee713-45de-4031-9f59-3f01260cc159`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:20,731::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:44:20,744::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:20,744::task::1191::Storage.TaskManager.Task::(prepare) Task=`9a2ee713-45de-4031-9f59-3f01260cc159`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:20,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a2ee713-45de-4031-9f59-3f01260cc159`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:20,745::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:20,745::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:20,745::task::993::Storage.TaskManager.Task::(_decref) Task=`9a2ee713-45de-4031-9f59-3f01260cc159`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:44:20,757::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:44:20,757::task::595::Storage.TaskManager.Task::(_updateState) Task=`1899736b-5603-421d-818d-cbeeb1aa3b03`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:44:20,758::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:44:20,758::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e140264d-d9fa-46ed-961b-f72ad83af992`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:44:20,759::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:44:20,759::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:44:20,759::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e140264d-d9fa-46ed-961b-f72ad83af992`::Granted request >Thread-34::DEBUG::2014-08-11 15:44:20,760::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1899736b-5603-421d-818d-cbeeb1aa3b03`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:44:20,760::task::993::Storage.TaskManager.Task::(_decref) Task=`1899736b-5603-421d-818d-cbeeb1aa3b03`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:20,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:20,777::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:20,778::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:44:20,780::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:20,780::task::1191::Storage.TaskManager.Task::(prepare) Task=`1899736b-5603-421d-818d-cbeeb1aa3b03`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:20,781::task::595::Storage.TaskManager.Task::(_updateState) Task=`1899736b-5603-421d-818d-cbeeb1aa3b03`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:44:20,781::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:44:20,781::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:44:20,781::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:44:20,782::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:44:20,782::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:44:20,782::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:44:20,783::task::993::Storage.TaskManager.Task::(_decref) Task=`1899736b-5603-421d-818d-cbeeb1aa3b03`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:44:20,792::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:20,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0155971 s, 65.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:22,851::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:22,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131816 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:24,541::__init__::232::IOProcess::(_processLogs) (7614) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:24,542::__init__::232::IOProcess::(_processLogs) (7614) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:44:24,543::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:44:24,544::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:24,563::__init__::232::IOProcess::(_processLogs) (7615) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:24,563::__init__::232::IOProcess::(_processLogs) (7615) Queuing response >Thread-16::DEBUG::2014-08-11 15:44:24,564::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000842182 s, 540 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:24,565::__init__::232::IOProcess::(_processLogs) (7614) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:24,566::__init__::232::IOProcess::(_processLogs) (7614) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:24,566::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:24,566::__init__::232::IOProcess::(_processLogs) (7616) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:24,567::__init__::232::IOProcess::(_processLogs) DEBUG|(7616) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:24,568::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:24,569::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:24,570::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:24,570::__init__::232::IOProcess::(_processLogs) (7617) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:24,570::__init__::232::IOProcess::(_processLogs) (7617) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:24,571::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:24,571::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:24,572::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:24,572::__init__::232::IOProcess::(_processLogs) (7618) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:24,572::__init__::232::IOProcess::(_processLogs) (7618) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:24,573::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:24,573::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:24,574::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:24,575::__init__::232::IOProcess::(_processLogs) (7619) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:24,575::__init__::232::IOProcess::(_processLogs) (7619) Queuing response >Dummy-33::DEBUG::2014-08-11 15:44:24,907::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:24,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131805 s, 77.7 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:44:25,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff24afc5-9c2f-4b78-a738-a882fde0b0dc`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:44:25,429::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:44:25,432::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:25,432::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:25,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:25,433::__init__::232::IOProcess::(_processLogs) (7620) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:44:25,433::__init__::232::IOProcess::(_processLogs) DEBUG|(7620) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:25,434::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:25,435::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:25,435::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:25,435::__init__::232::IOProcess::(_processLogs) (7621) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:44:25,436::__init__::232::IOProcess::(_processLogs) DEBUG|(7621) Queuing response >Thread-122::INFO::2014-08-11 15:44:25,436::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:44:25,436::task::1191::Storage.TaskManager.Task::(prepare) Task=`ff24afc5-9c2f-4b78-a738-a882fde0b0dc`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:44:25,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff24afc5-9c2f-4b78-a738-a882fde0b0dc`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:44:25,437::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:44:25,437::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:44:25,437::task::993::Storage.TaskManager.Task::(_decref) Task=`ff24afc5-9c2f-4b78-a738-a882fde0b0dc`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:25,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:25,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:25,868::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:26,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:26,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132333 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:29,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:29,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133085 s, 76.9 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:44:30,800::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:44:30,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`392dbbe5-f14d-4bf8-a54d-d880410fc205`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:30,801::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:44:30,814::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:30,815::task::1191::Storage.TaskManager.Task::(prepare) Task=`392dbbe5-f14d-4bf8-a54d-d880410fc205`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:30,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`392dbbe5-f14d-4bf8-a54d-d880410fc205`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:30,815::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:30,815::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:30,816::task::993::Storage.TaskManager.Task::(_decref) Task=`392dbbe5-f14d-4bf8-a54d-d880410fc205`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:44:30,826::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:44:30,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`3653df9e-1148-4183-9144-a01c03966790`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:44:30,827::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:44:30,828::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`71a843e1-eff1-4dfe-8f2a-ffcdcea9b651`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:44:30,828::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:44:30,828::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:44:30,828::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`71a843e1-eff1-4dfe-8f2a-ffcdcea9b651`::Granted request >Thread-34::DEBUG::2014-08-11 15:44:30,829::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3653df9e-1148-4183-9144-a01c03966790`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:44:30,829::task::993::Storage.TaskManager.Task::(_decref) Task=`3653df9e-1148-4183-9144-a01c03966790`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:44:30,849::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:30,850::task::1191::Storage.TaskManager.Task::(prepare) Task=`3653df9e-1148-4183-9144-a01c03966790`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:30,850::task::595::Storage.TaskManager.Task::(_updateState) Task=`3653df9e-1148-4183-9144-a01c03966790`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:44:30,850::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:44:30,850::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:44:30,851::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:44:30,851::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:44:30,851::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:44:30,851::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:44:30,852::task::993::Storage.TaskManager.Task::(_decref) Task=`3653df9e-1148-4183-9144-a01c03966790`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:30,875::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:30,876::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:30,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:31,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:31,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133254 s, 76.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:44:31,759::task::595::Storage.TaskManager.Task::(_updateState) Task=`3aefcdaa-b73a-4c77-9189-db5dd3997ad3`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:31,760::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:44:31,760::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000842182', 'lastCheck': '7.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:31,761::task::1191::Storage.TaskManager.Task::(prepare) Task=`3aefcdaa-b73a-4c77-9189-db5dd3997ad3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000842182', 'lastCheck': '7.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:31,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`3aefcdaa-b73a-4c77-9189-db5dd3997ad3`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:31,761::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:31,761::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:31,761::task::993::Storage.TaskManager.Task::(_decref) Task=`3aefcdaa-b73a-4c77-9189-db5dd3997ad3`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:44:31,781::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:33,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:33,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134175 s, 76.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:34,575::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:34,576::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:34,576::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:34,576::__init__::232::IOProcess::(_processLogs) (7622) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:34,577::__init__::232::IOProcess::(_processLogs) DEBUG|(7622) Queuing response >Thread-16::DEBUG::2014-08-11 15:44:34,577::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:44:34,597::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421791 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:34,599::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:34,599::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:34,599::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:34,600::__init__::232::IOProcess::(_processLogs) (7623) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:34,600::__init__::232::IOProcess::(_processLogs) DEBUG|(7623) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:34,602::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:34,602::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:34,602::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:34,603::__init__::232::IOProcess::(_processLogs) (7624) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:34,603::__init__::232::IOProcess::(_processLogs) DEBUG|(7624) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:34,604::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:34,604::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:34,605::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:34,605::__init__::232::IOProcess::(_processLogs) (7625) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:34,606::__init__::232::IOProcess::(_processLogs) (7625) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:34,606::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:34,606::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:34,607::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:34,607::__init__::232::IOProcess::(_processLogs) (7626) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:34,607::__init__::232::IOProcess::(_processLogs) (7626) Queuing response >Dummy-33::DEBUG::2014-08-11 15:44:35,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:35,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132513 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:35,884::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:35,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:35,969::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:37,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:37,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01334 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:39,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:39,337::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131843 s, 77.7 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:44:40,863::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:44:40,864::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6715fa5-778e-4720-81ee-9cd948e3b928`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:40,865::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:44:40,878::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:40,879::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6715fa5-778e-4720-81ee-9cd948e3b928`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:40,879::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6715fa5-778e-4720-81ee-9cd948e3b928`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:40,879::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:40,879::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:40,879::task::993::Storage.TaskManager.Task::(_decref) Task=`f6715fa5-778e-4720-81ee-9cd948e3b928`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:44:40,889::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:44:40,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5e8cbb0-44ff-4f01-8676-529d24f46007`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:44:40,970::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:44:40,970::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4101f722-9e11-4ba0-bff7-cd561711c35c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:44:40,971::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:44:40,971::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:44:40,971::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4101f722-9e11-4ba0-bff7-cd561711c35c`::Granted request >Thread-34::DEBUG::2014-08-11 15:44:40,972::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b5e8cbb0-44ff-4f01-8676-529d24f46007`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:44:40,972::task::993::Storage.TaskManager.Task::(_decref) Task=`b5e8cbb0-44ff-4f01-8676-529d24f46007`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:40,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:40,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:40,988::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:44:41,007::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:41,007::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5e8cbb0-44ff-4f01-8676-529d24f46007`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:41,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5e8cbb0-44ff-4f01-8676-529d24f46007`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:44:41,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:44:41,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:44:41,008::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:44:41,008::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:44:41,009::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:44:41,009::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:44:41,009::task::993::Storage.TaskManager.Task::(_decref) Task=`b5e8cbb0-44ff-4f01-8676-529d24f46007`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:44:41,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:41,393::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131722 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:43,415::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:43,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132046 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:44,609::__init__::232::IOProcess::(_processLogs) (7626) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:44,610::__init__::232::IOProcess::(_processLogs) (7626) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:44:44,611::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:44:44,612::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:44,631::__init__::232::IOProcess::(_processLogs) (7627) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:44,632::__init__::232::IOProcess::(_processLogs) (7627) Queuing response >Thread-16::DEBUG::2014-08-11 15:44:44,633::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00091765 s, 496 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:44,634::__init__::232::IOProcess::(_processLogs) (7626) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:44:44,634::__init__::232::IOProcess::(_processLogs) (7626) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:44:44,634::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:44,635::__init__::232::IOProcess::(_processLogs) (7628) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:44,636::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:44,636::__init__::232::IOProcess::(_processLogs) (7629) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:44,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:44,637::__init__::232::IOProcess::(_processLogs) (7630) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:44,638::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:44,638::__init__::232::IOProcess::(_processLogs) (7631) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:44,639::__init__::232::IOProcess::(_processLogs) (7631) Queuing response >Dummy-33::DEBUG::2014-08-11 15:44:45,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:45,509::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0146349 s, 70.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:45,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:46,000::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:46,071::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:44:46,884::task::595::Storage.TaskManager.Task::(_updateState) Task=`425f3438-d3ad-4c9e-9de8-64cf280fdc9f`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:46,884::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:44:46,884::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00091765', 'lastCheck': '2.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:46,885::task::1191::Storage.TaskManager.Task::(prepare) Task=`425f3438-d3ad-4c9e-9de8-64cf280fdc9f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00091765', 'lastCheck': '2.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:44:46,885::task::595::Storage.TaskManager.Task::(_updateState) Task=`425f3438-d3ad-4c9e-9de8-64cf280fdc9f`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:46,885::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:46,885::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:46,885::task::993::Storage.TaskManager.Task::(_decref) Task=`425f3438-d3ad-4c9e-9de8-64cf280fdc9f`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:44:46,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:47,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:47,566::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134103 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:49,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:49,624::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132775 s, 77.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:44:51,022::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:44:51,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4bcb316-2563-4683-919e-8e1585fe1e46`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:44:51,070::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:51,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:51,080::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:51,081::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:44:51,083::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:51,084::task::1191::Storage.TaskManager.Task::(prepare) Task=`d4bcb316-2563-4683-919e-8e1585fe1e46`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:44:51,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`d4bcb316-2563-4683-919e-8e1585fe1e46`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:44:51,084::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:44:51,084::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:44:51,084::task::993::Storage.TaskManager.Task::(_decref) Task=`d4bcb316-2563-4683-919e-8e1585fe1e46`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:44:51,095::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:44:51,096::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea43d497-2851-44a8-bf59-0bad9942ec73`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:44:51,096::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:44:51,097::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5abfa49c-edcf-4f48-9a1f-d6399bf3d312`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:44:51,097::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:44:51,097::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:44:51,098::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5abfa49c-edcf-4f48-9a1f-d6399bf3d312`::Granted request >Thread-34::DEBUG::2014-08-11 15:44:51,098::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ea43d497-2851-44a8-bf59-0bad9942ec73`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:44:51,098::task::993::Storage.TaskManager.Task::(_decref) Task=`ea43d497-2851-44a8-bf59-0bad9942ec73`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:44:51,118::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:51,119::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea43d497-2851-44a8-bf59-0bad9942ec73`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:44:51,119::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea43d497-2851-44a8-bf59-0bad9942ec73`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:44:51,119::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:44:51,119::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:44:51,120::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:44:51,120::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:44:51,120::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:44:51,120::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:44:51,121::task::993::Storage.TaskManager.Task::(_decref) Task=`ea43d497-2851-44a8-bf59-0bad9942ec73`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:44:51,646::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:51,680::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133161 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:53,702::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:53,736::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013247 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:54,641::__init__::232::IOProcess::(_processLogs) (7632) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:44:54,642::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:44:54,661::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418614 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:44:54,663::__init__::232::IOProcess::(_processLogs) (7633) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:44:54,664::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:54,664::__init__::232::IOProcess::(_processLogs) (7634) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:54,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:54,665::__init__::232::IOProcess::(_processLogs) (7635) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:44:54,666::__init__::232::IOProcess::(_processLogs) (7635) Queuing response >Thread-14::DEBUG::2014-08-11 15:44:54,667::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:44:54,667::__init__::232::IOProcess::(_processLogs) (7636) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:44:55,759::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:55,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0168784 s, 60.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:56,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:56,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:44:56,171::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:44:57,830::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:57,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131865 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:44:59,886::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:44:59,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133871 s, 76.5 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:01,132::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:45:01,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`a11ffe5c-b51a-4965-bd7d-6ef04639dd8b`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:01,171::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:01,178::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:01,181::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:01,184::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:45:01,184::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:01,186::task::1191::Storage.TaskManager.Task::(prepare) Task=`a11ffe5c-b51a-4965-bd7d-6ef04639dd8b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:01,186::task::595::Storage.TaskManager.Task::(_updateState) Task=`a11ffe5c-b51a-4965-bd7d-6ef04639dd8b`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:01,187::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:01,187::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:01,187::task::993::Storage.TaskManager.Task::(_decref) Task=`a11ffe5c-b51a-4965-bd7d-6ef04639dd8b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:45:01,197::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:45:01,198::task::595::Storage.TaskManager.Task::(_updateState) Task=`48061d72-4a22-4273-9be6-7045b9c8f148`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:45:01,198::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:45:01,199::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3710c963-c1c4-4fae-92e4-45559d499838`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:45:01,199::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:45:01,200::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:45:01,200::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3710c963-c1c4-4fae-92e4-45559d499838`::Granted request >Thread-34::DEBUG::2014-08-11 15:45:01,201::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`48061d72-4a22-4273-9be6-7045b9c8f148`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:45:01,201::task::993::Storage.TaskManager.Task::(_decref) Task=`48061d72-4a22-4273-9be6-7045b9c8f148`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:45:01,221::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:01,222::task::1191::Storage.TaskManager.Task::(prepare) Task=`48061d72-4a22-4273-9be6-7045b9c8f148`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:01,222::task::595::Storage.TaskManager.Task::(_updateState) Task=`48061d72-4a22-4273-9be6-7045b9c8f148`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:45:01,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:45:01,222::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:45:01,223::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:45:01,223::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:45:01,223::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:45:01,224::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:45:01,224::task::993::Storage.TaskManager.Task::(_decref) Task=`48061d72-4a22-4273-9be6-7045b9c8f148`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:45:01,943::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:01,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.028122 s, 36.4 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:02,011::task::595::Storage.TaskManager.Task::(_updateState) Task=`10077ada-eefc-4656-bcaa-4eecce2e73da`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:02,013::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:45:02,015::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418614', 'lastCheck': '7.3', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:02,016::task::1191::Storage.TaskManager.Task::(prepare) Task=`10077ada-eefc-4656-bcaa-4eecce2e73da`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418614', 'lastCheck': '7.3', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:02,018::task::595::Storage.TaskManager.Task::(_updateState) Task=`10077ada-eefc-4656-bcaa-4eecce2e73da`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:02,019::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:02,020::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:02,021::task::993::Storage.TaskManager.Task::(_decref) Task=`10077ada-eefc-4656-bcaa-4eecce2e73da`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:45:02,083::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:04,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:04,291::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132363 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:04,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:45:04,670::__init__::232::IOProcess::(_processLogs) (7637) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:45:04,671::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:45:04,691::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423084 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:04,692::__init__::232::IOProcess::(_processLogs) (7638) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:45:04,694::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:45:04,694::__init__::232::IOProcess::(_processLogs) (7639) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:04,695::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:45:04,695::__init__::232::IOProcess::(_processLogs) (7640) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:04,696::__init__::232::IOProcess::(_processLogs) (7640) Queuing response >Thread-14::DEBUG::2014-08-11 15:45:04,697::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:45:04,697::__init__::232::IOProcess::(_processLogs) (7641) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:04,697::__init__::232::IOProcess::(_processLogs) (7641) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:06,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:06,193::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:06,272::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:06,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:06,347::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132976 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:08,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:08,404::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133298 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:10,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:10,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0152197 s, 67.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:11,235::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:45:11,236::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7390acd-51e5-449f-9387-95c4744bcc06`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:11,271::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:11,279::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:11,281::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:11,282::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:45:11,284::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:11,285::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7390acd-51e5-449f-9387-95c4744bcc06`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:11,285::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7390acd-51e5-449f-9387-95c4744bcc06`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:11,285::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:11,285::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:11,285::task::993::Storage.TaskManager.Task::(_decref) Task=`e7390acd-51e5-449f-9387-95c4744bcc06`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:45:11,296::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:45:11,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e832070-f886-4d1b-8ef1-7a16ddd72f43`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:45:11,297::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:45:11,297::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f6391f38-a486-44ba-9bb2-6bae321d8eac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:45:11,298::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:45:11,298::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:45:11,298::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f6391f38-a486-44ba-9bb2-6bae321d8eac`::Granted request >Thread-34::DEBUG::2014-08-11 15:45:11,299::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5e832070-f886-4d1b-8ef1-7a16ddd72f43`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:45:11,299::task::993::Storage.TaskManager.Task::(_decref) Task=`5e832070-f886-4d1b-8ef1-7a16ddd72f43`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:45:11,319::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:11,319::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e832070-f886-4d1b-8ef1-7a16ddd72f43`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:11,319::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e832070-f886-4d1b-8ef1-7a16ddd72f43`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:45:11,319::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:45:11,320::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:45:11,320::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:45:11,321::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:45:11,321::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:45:11,321::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:45:11,322::task::993::Storage.TaskManager.Task::(_decref) Task=`5e832070-f886-4d1b-8ef1-7a16ddd72f43`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:45:12,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:12,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131608 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:14,542::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:14,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136922 s, 74.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:14,699::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:45:14,699::__init__::232::IOProcess::(_processLogs) (7642) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:45:14,700::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:45:14,720::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418693 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:14,721::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:45:14,722::__init__::232::IOProcess::(_processLogs) (7643) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:45:14,723::__init__::232::IOProcess::(_processLogs) (7644) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:14,724::__init__::232::IOProcess::(_processLogs) (7645) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:14,725::__init__::232::IOProcess::(_processLogs) (7646) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:16,289::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:16,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:16,372::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:16,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:16,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133297 s, 76.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:17,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e25fc9a-7a54-47f6-aa3d-f4e0a453d928`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:17,223::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:45:17,224::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418693', 'lastCheck': '2.5', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:17,224::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e25fc9a-7a54-47f6-aa3d-f4e0a453d928`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418693', 'lastCheck': '2.5', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:17,224::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e25fc9a-7a54-47f6-aa3d-f4e0a453d928`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:17,225::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:17,225::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:17,225::task::993::Storage.TaskManager.Task::(_decref) Task=`9e25fc9a-7a54-47f6-aa3d-f4e0a453d928`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:45:17,245::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:18,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:18,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132937 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:20,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:20,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132381 s, 77.4 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:21,334::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:45:21,335::task::595::Storage.TaskManager.Task::(_updateState) Task=`15522747-5be4-4331-831a-54b0af8427ac`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:21,372::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:21,380::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:21,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:21,382::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:45:21,384::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:21,385::task::1191::Storage.TaskManager.Task::(prepare) Task=`15522747-5be4-4331-831a-54b0af8427ac`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:21,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`15522747-5be4-4331-831a-54b0af8427ac`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:21,385::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:21,385::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:21,385::task::993::Storage.TaskManager.Task::(_decref) Task=`15522747-5be4-4331-831a-54b0af8427ac`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:45:21,396::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:45:21,396::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fe2b1cb-109e-4db2-8fb5-6ff43b7e94ed`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:45:21,397::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:45:21,397::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1b951537-7ec2-427e-9072-8aaef4784aac`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:45:21,398::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:45:21,398::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:45:21,398::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1b951537-7ec2-427e-9072-8aaef4784aac`::Granted request >Thread-34::DEBUG::2014-08-11 15:45:21,399::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8fe2b1cb-109e-4db2-8fb5-6ff43b7e94ed`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:45:21,399::task::993::Storage.TaskManager.Task::(_decref) Task=`8fe2b1cb-109e-4db2-8fb5-6ff43b7e94ed`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:45:21,419::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:21,419::task::1191::Storage.TaskManager.Task::(prepare) Task=`8fe2b1cb-109e-4db2-8fb5-6ff43b7e94ed`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:21,419::task::595::Storage.TaskManager.Task::(_updateState) Task=`8fe2b1cb-109e-4db2-8fb5-6ff43b7e94ed`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:45:21,420::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:45:21,420::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:45:21,420::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:45:21,420::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:45:21,421::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:45:21,421::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:45:21,421::task::993::Storage.TaskManager.Task::(_decref) Task=`8fe2b1cb-109e-4db2-8fb5-6ff43b7e94ed`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:45:22,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:22,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130815 s, 78.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:24,727::__init__::232::IOProcess::(_processLogs) (7647) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:45:24,728::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:45:24,748::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416072 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:24,749::__init__::232::IOProcess::(_processLogs) (7648) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:45:24,751::__init__::232::IOProcess::(_processLogs) (7649) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:24,752::__init__::232::IOProcess::(_processLogs) (7650) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:24,753::__init__::232::IOProcess::(_processLogs) (7651) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:45:24,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:24,857::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132788 s, 77.1 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:45:25,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`51d493a8-bfbd-45f9-a8d4-710232a9d9b8`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:45:25,541::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:45:25,542::__init__::232::IOProcess::(_processLogs) (7652) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:45:25,544::__init__::232::IOProcess::(_processLogs) (7653) Got request for method 'stat' >Thread-122::INFO::2014-08-11 15:45:25,545::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:45:25,545::task::1191::Storage.TaskManager.Task::(prepare) Task=`51d493a8-bfbd-45f9-a8d4-710232a9d9b8`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:45:25,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`51d493a8-bfbd-45f9-a8d4-710232a9d9b8`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:45:25,546::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:45:25,546::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:45:25,546::task::993::Storage.TaskManager.Task::(_decref) Task=`51d493a8-bfbd-45f9-a8d4-710232a9d9b8`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:26,389::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:26,390::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:26,472::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:26,880::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:26,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013323 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:28,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:28,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131576 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:30,992::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:31,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133329 s, 76.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:31,435::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:45:31,436::task::595::Storage.TaskManager.Task::(_updateState) Task=`27d07380-5b84-45fc-8819-5de499b63deb`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:31,472::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:31,480::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:31,482::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:31,484::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:45:31,486::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:31,486::task::1191::Storage.TaskManager.Task::(prepare) Task=`27d07380-5b84-45fc-8819-5de499b63deb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:31,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`27d07380-5b84-45fc-8819-5de499b63deb`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:31,487::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:31,487::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:31,487::task::993::Storage.TaskManager.Task::(_decref) Task=`27d07380-5b84-45fc-8819-5de499b63deb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:45:31,499::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:45:31,500::task::595::Storage.TaskManager.Task::(_updateState) Task=`3331600f-49f9-44ba-aa29-b0340f3722cb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:45:31,500::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:45:31,501::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`253eae46-8760-43e1-b091-e426bc5e1ce6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:45:31,501::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:45:31,502::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:45:31,502::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`253eae46-8760-43e1-b091-e426bc5e1ce6`::Granted request >Thread-34::DEBUG::2014-08-11 15:45:31,503::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3331600f-49f9-44ba-aa29-b0340f3722cb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:45:31,503::task::993::Storage.TaskManager.Task::(_decref) Task=`3331600f-49f9-44ba-aa29-b0340f3722cb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:45:31,523::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:31,524::task::1191::Storage.TaskManager.Task::(prepare) Task=`3331600f-49f9-44ba-aa29-b0340f3722cb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:31,524::task::595::Storage.TaskManager.Task::(_updateState) Task=`3331600f-49f9-44ba-aa29-b0340f3722cb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:45:31,524::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:45:31,524::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:45:31,525::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:45:31,525::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:45:31,525::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:45:31,526::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:45:31,526::task::993::Storage.TaskManager.Task::(_decref) Task=`3331600f-49f9-44ba-aa29-b0340f3722cb`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:45:32,336::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ea9ff37-8ef7-4ba6-ae25-6cf6a26dcc2b`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:32,337::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:45:32,340::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416072', 'lastCheck': '7.6', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:32,342::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ea9ff37-8ef7-4ba6-ae25-6cf6a26dcc2b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416072', 'lastCheck': '7.6', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:32,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ea9ff37-8ef7-4ba6-ae25-6cf6a26dcc2b`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:32,344::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:32,346::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:32,348::task::993::Storage.TaskManager.Task::(_decref) Task=`4ea9ff37-8ef7-4ba6-ae25-6cf6a26dcc2b`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:45:32,399::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:33,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:33,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132053 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:34,755::__init__::232::IOProcess::(_processLogs) (7654) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:45:34,756::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:45:34,776::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423567 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:34,777::__init__::232::IOProcess::(_processLogs) (7655) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:45:34,778::__init__::232::IOProcess::(_processLogs) (7656) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:34,779::__init__::232::IOProcess::(_processLogs) (7657) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:34,780::__init__::232::IOProcess::(_processLogs) (7658) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:45:35,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:35,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133332 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:36,491::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:36,493::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:36,572::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:37,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:37,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131745 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:39,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:39,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132685 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:41,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:41,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132415 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:41,539::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:45:41,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`b05024e1-bb3d-4d64-b1df-bd1bdc85a80c`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:41,572::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:41,579::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:41,581::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:41,582::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:45:41,585::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:41,585::task::1191::Storage.TaskManager.Task::(prepare) Task=`b05024e1-bb3d-4d64-b1df-bd1bdc85a80c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:41,586::task::595::Storage.TaskManager.Task::(_updateState) Task=`b05024e1-bb3d-4d64-b1df-bd1bdc85a80c`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:41,586::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:41,586::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:41,586::task::993::Storage.TaskManager.Task::(_decref) Task=`b05024e1-bb3d-4d64-b1df-bd1bdc85a80c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:45:41,596::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:45:41,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`c518fc1d-84cd-4c33-94fc-2f4716a5dc71`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:45:41,597::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:45:41,598::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1d6df9c7-8c8e-4da4-bde4-e3cfc6b38a21`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:45:41,598::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:45:41,599::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:45:41,599::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1d6df9c7-8c8e-4da4-bde4-e3cfc6b38a21`::Granted request >Thread-34::DEBUG::2014-08-11 15:45:41,599::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c518fc1d-84cd-4c33-94fc-2f4716a5dc71`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:45:41,600::task::993::Storage.TaskManager.Task::(_decref) Task=`c518fc1d-84cd-4c33-94fc-2f4716a5dc71`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:45:41,619::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:41,620::task::1191::Storage.TaskManager.Task::(prepare) Task=`c518fc1d-84cd-4c33-94fc-2f4716a5dc71`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:41,620::task::595::Storage.TaskManager.Task::(_updateState) Task=`c518fc1d-84cd-4c33-94fc-2f4716a5dc71`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:45:41,620::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:45:41,620::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:45:41,621::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:45:41,621::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:45:41,621::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:45:41,621::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:45:41,622::task::993::Storage.TaskManager.Task::(_decref) Task=`c518fc1d-84cd-4c33-94fc-2f4716a5dc71`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:45:43,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:43,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132842 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:44,784::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:45:44,785::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:45:44,786::__init__::232::IOProcess::(_processLogs) (7659) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:45:44,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:45:44,807::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000431413 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:44,809::__init__::232::IOProcess::(_processLogs) (7660) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:45:44,810::__init__::232::IOProcess::(_processLogs) (7661) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:44,811::__init__::232::IOProcess::(_processLogs) (7662) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:44,812::__init__::232::IOProcess::(_processLogs) (7663) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:45:45,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:45,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132186 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:46,589::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:46,590::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:46,591::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:47,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:47,477::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132328 s, 77.4 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:45:47,504::task::595::Storage.TaskManager.Task::(_updateState) Task=`82524347-bd78-4994-aed8-4b767ec1f7e5`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:47,505::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:45:47,505::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000431413', 'lastCheck': '2.7', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:47,506::task::1191::Storage.TaskManager.Task::(prepare) Task=`82524347-bd78-4994-aed8-4b767ec1f7e5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000431413', 'lastCheck': '2.7', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:45:47,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`82524347-bd78-4994-aed8-4b767ec1f7e5`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:47,506::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:47,506::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:47,507::task::993::Storage.TaskManager.Task::(_decref) Task=`82524347-bd78-4994-aed8-4b767ec1f7e5`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:45:47,528::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:49,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:49,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013607 s, 75.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:51,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:51,590::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133145 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:51,600::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:45:51,637::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:45:51,674::task::595::Storage.TaskManager.Task::(_updateState) Task=`e822fd5f-f92f-4679-8cd6-d725beca5224`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:45:51,676::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:51,676::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:51,679::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:45:51,690::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:51,691::task::1191::Storage.TaskManager.Task::(prepare) Task=`e822fd5f-f92f-4679-8cd6-d725beca5224`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:45:51,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`e822fd5f-f92f-4679-8cd6-d725beca5224`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:45:51,691::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:45:51,691::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:45:51,692::task::993::Storage.TaskManager.Task::(_decref) Task=`e822fd5f-f92f-4679-8cd6-d725beca5224`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:45:51,703::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:45:51,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`02d19dc5-6b59-4bfb-b779-3b965a8b64a3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:45:51,704::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:45:51,705::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0399e3b4-a8ac-4a7b-8ae5-b7327852d61a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:45:51,705::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:45:51,705::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:45:51,706::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0399e3b4-a8ac-4a7b-8ae5-b7327852d61a`::Granted request >Thread-34::DEBUG::2014-08-11 15:45:51,706::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`02d19dc5-6b59-4bfb-b779-3b965a8b64a3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:45:51,706::task::993::Storage.TaskManager.Task::(_decref) Task=`02d19dc5-6b59-4bfb-b779-3b965a8b64a3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:45:51,727::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:51,728::task::1191::Storage.TaskManager.Task::(prepare) Task=`02d19dc5-6b59-4bfb-b779-3b965a8b64a3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:45:51,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`02d19dc5-6b59-4bfb-b779-3b965a8b64a3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:45:51,728::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:45:51,728::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:45:51,729::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:45:51,729::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:45:51,729::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:45:51,730::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:45:51,730::task::993::Storage.TaskManager.Task::(_decref) Task=`02d19dc5-6b59-4bfb-b779-3b965a8b64a3`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:45:53,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:53,649::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013165 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:54,814::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:45:54,815::__init__::232::IOProcess::(_processLogs) (7664) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:45:54,816::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:45:54,835::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000434258 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:45:54,837::__init__::232::IOProcess::(_processLogs) (7665) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:45:54,838::__init__::232::IOProcess::(_processLogs) (7666) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:54,839::__init__::232::IOProcess::(_processLogs) (7667) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:45:54,840::__init__::232::IOProcess::(_processLogs) (7668) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:45:55,671::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:55,705::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132154 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:56,686::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:56,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:45:56,773::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:45:57,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:57,762::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0142099 s, 72.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:45:59,794::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:45:59,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0240546 s, 42.6 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:01,743::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:46:01,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1609b9e-2d86-4a6b-b173-9b656910daaa`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:01,773::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:01,780::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:01,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:01,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:46:01,785::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:01,786::task::1191::Storage.TaskManager.Task::(prepare) Task=`c1609b9e-2d86-4a6b-b173-9b656910daaa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:01,786::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1609b9e-2d86-4a6b-b173-9b656910daaa`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:01,786::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:01,786::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:01,787::task::993::Storage.TaskManager.Task::(_decref) Task=`c1609b9e-2d86-4a6b-b173-9b656910daaa`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:46:01,797::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:46:01,798::task::595::Storage.TaskManager.Task::(_updateState) Task=`86512181-53f8-4d5c-b0b7-51e842ed6aa1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:46:01,798::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:46:01,799::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bbf0aa43-0168-4bc5-b35e-8950546ae30f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:46:01,799::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:46:01,800::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:46:01,800::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`bbf0aa43-0168-4bc5-b35e-8950546ae30f`::Granted request >Thread-34::DEBUG::2014-08-11 15:46:01,800::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`86512181-53f8-4d5c-b0b7-51e842ed6aa1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:46:01,801::task::993::Storage.TaskManager.Task::(_decref) Task=`86512181-53f8-4d5c-b0b7-51e842ed6aa1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:46:01,821::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:01,821::task::1191::Storage.TaskManager.Task::(prepare) Task=`86512181-53f8-4d5c-b0b7-51e842ed6aa1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:01,822::task::595::Storage.TaskManager.Task::(_updateState) Task=`86512181-53f8-4d5c-b0b7-51e842ed6aa1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:46:01,822::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:46:01,822::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:46:01,822::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:46:01,823::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:46:01,823::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:46:01,823::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:46:01,824::task::993::Storage.TaskManager.Task::(_decref) Task=`86512181-53f8-4d5c-b0b7-51e842ed6aa1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:46:01,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:01,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013247 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:02,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b83d988-6a2a-4bcb-99ff-16a6a785ec0d`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:02,625::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:46:02,627::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000434258', 'lastCheck': '7.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:02,629::task::1191::Storage.TaskManager.Task::(prepare) Task=`0b83d988-6a2a-4bcb-99ff-16a6a785ec0d`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000434258', 'lastCheck': '7.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:02,630::task::595::Storage.TaskManager.Task::(_updateState) Task=`0b83d988-6a2a-4bcb-99ff-16a6a785ec0d`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:02,631::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:02,632::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:02,633::task::993::Storage.TaskManager.Task::(_decref) Task=`0b83d988-6a2a-4bcb-99ff-16a6a785ec0d`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:46:02,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:03,930::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:03,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013223 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:04,843::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:04,843::__init__::232::IOProcess::(_processLogs) (7669) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:46:04,844::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:46:04,864::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425626 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:04,866::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:04,866::__init__::232::IOProcess::(_processLogs) (7670) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:04,867::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:04,868::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:04,868::__init__::232::IOProcess::(_processLogs) (7671) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:04,868::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7671) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:04,869::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:04,870::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:04,871::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:04,871::__init__::232::IOProcess::(_processLogs) (7672) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:04,871::__init__::232::IOProcess::(_processLogs) (7672) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:04,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:04,875::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:04,888::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:04,889::__init__::232::IOProcess::(_processLogs) (7673) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:04,889::__init__::232::IOProcess::(_processLogs) (7673) Queuing response >Dummy-33::DEBUG::2014-08-11 15:46:05,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:06,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132315 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:06,788::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:06,789::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:06,874::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:08,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:08,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132688 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:10,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:10,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131029 s, 78.2 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:11,835::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:46:11,836::task::595::Storage.TaskManager.Task::(_updateState) Task=`836f51f1-5d81-4760-9fbc-5b1bd95045c7`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:11,874::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:11,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:11,883::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:11,884::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:46:11,886::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:11,887::task::1191::Storage.TaskManager.Task::(prepare) Task=`836f51f1-5d81-4760-9fbc-5b1bd95045c7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:11,887::task::595::Storage.TaskManager.Task::(_updateState) Task=`836f51f1-5d81-4760-9fbc-5b1bd95045c7`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:11,887::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:11,887::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:11,887::task::993::Storage.TaskManager.Task::(_decref) Task=`836f51f1-5d81-4760-9fbc-5b1bd95045c7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:46:11,897::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:46:11,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`26c64ab5-e44d-4fee-a7aa-ac84f269af8c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:46:11,898::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:46:11,899::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6ac7cf1a-8746-4ed9-b8c2-ad3a522205e6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:46:11,899::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:46:11,900::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:46:11,900::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6ac7cf1a-8746-4ed9-b8c2-ad3a522205e6`::Granted request >Thread-34::DEBUG::2014-08-11 15:46:11,902::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`26c64ab5-e44d-4fee-a7aa-ac84f269af8c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:46:11,902::task::993::Storage.TaskManager.Task::(_decref) Task=`26c64ab5-e44d-4fee-a7aa-ac84f269af8c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:46:11,922::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:11,923::task::1191::Storage.TaskManager.Task::(prepare) Task=`26c64ab5-e44d-4fee-a7aa-ac84f269af8c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:11,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`26c64ab5-e44d-4fee-a7aa-ac84f269af8c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:46:11,923::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:46:11,923::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:46:11,924::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:46:11,924::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:46:11,924::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:46:11,924::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:46:11,925::task::993::Storage.TaskManager.Task::(_decref) Task=`26c64ab5-e44d-4fee-a7aa-ac84f269af8c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:46:12,170::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:12,205::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134638 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:14,228::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:14,261::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132281 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:14,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:14,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:14,890::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:14,890::__init__::232::IOProcess::(_processLogs) (7674) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:14,890::__init__::232::IOProcess::(_processLogs) (7674) Queuing response >Thread-16::DEBUG::2014-08-11 15:46:14,891::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:46:14,911::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421839 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:14,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:14,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:14,913::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:14,913::__init__::232::IOProcess::(_processLogs) (7675) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:14,913::__init__::232::IOProcess::(_processLogs) (7675) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:14,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:14,915::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:14,915::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:14,916::__init__::232::IOProcess::(_processLogs) (7676) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:14,916::__init__::232::IOProcess::(_processLogs) (7676) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:14,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:14,917::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:14,918::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:14,918::__init__::232::IOProcess::(_processLogs) (7677) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:14,918::__init__::232::IOProcess::(_processLogs) (7677) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:14,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:14,919::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:14,920::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:14,920::__init__::232::IOProcess::(_processLogs) (7678) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:14,921::__init__::232::IOProcess::(_processLogs) (7678) Queuing response >Dummy-33::DEBUG::2014-08-11 15:46:16,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:16,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131803 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:16,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:16,891::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:16,974::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:46:17,792::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e5c0800-6209-4697-9a9d-84b39dcee429`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:17,792::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:46:17,793::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421839', 'lastCheck': '2.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:17,793::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e5c0800-6209-4697-9a9d-84b39dcee429`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421839', 'lastCheck': '2.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:17,793::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e5c0800-6209-4697-9a9d-84b39dcee429`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:17,793::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:17,794::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:17,794::task::993::Storage.TaskManager.Task::(_decref) Task=`8e5c0800-6209-4697-9a9d-84b39dcee429`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:46:17,814::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:18,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:18,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131676 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:20,397::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:20,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132534 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:21,938::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:46:21,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f1b80ef-f7a8-444c-9846-692d32bef4c9`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:21,973::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:21,981::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:21,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:21,984::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:46:21,986::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:21,986::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f1b80ef-f7a8-444c-9846-692d32bef4c9`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:21,987::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f1b80ef-f7a8-444c-9846-692d32bef4c9`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:21,987::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:21,987::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:21,987::task::993::Storage.TaskManager.Task::(_decref) Task=`5f1b80ef-f7a8-444c-9846-692d32bef4c9`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:46:21,998::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:46:21,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`954aeabd-22fb-49c1-9029-56fe4f6a94f9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:46:21,999::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:46:22,000::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`726280ec-a9f8-4fb4-b17f-046e2123a374`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:46:22,000::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:46:22,001::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:46:22,001::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`726280ec-a9f8-4fb4-b17f-046e2123a374`::Granted request >Thread-34::DEBUG::2014-08-11 15:46:22,001::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`954aeabd-22fb-49c1-9029-56fe4f6a94f9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:46:22,002::task::993::Storage.TaskManager.Task::(_decref) Task=`954aeabd-22fb-49c1-9029-56fe4f6a94f9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:46:22,021::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:22,022::task::1191::Storage.TaskManager.Task::(prepare) Task=`954aeabd-22fb-49c1-9029-56fe4f6a94f9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:22,022::task::595::Storage.TaskManager.Task::(_updateState) Task=`954aeabd-22fb-49c1-9029-56fe4f6a94f9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:46:22,022::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:46:22,022::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:46:22,023::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:46:22,023::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:46:22,023::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:46:22,023::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:46:22,024::task::993::Storage.TaskManager.Task::(_decref) Task=`954aeabd-22fb-49c1-9029-56fe4f6a94f9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:46:22,454::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:22,488::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136137 s, 75.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:24,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:24,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133202 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:24,921::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:24,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:24,922::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:24,922::__init__::232::IOProcess::(_processLogs) (7679) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:24,922::__init__::232::IOProcess::(_processLogs) (7679) Queuing response >Thread-16::DEBUG::2014-08-11 15:46:24,923::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:46:24,944::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000495739 s, 918 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:24,945::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:24,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:24,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:24,948::__init__::232::IOProcess::(_processLogs) (7680) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:24,948::__init__::232::IOProcess::(_processLogs) (7680) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:24,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:24,949::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:24,950::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:24,950::__init__::232::IOProcess::(_processLogs) (7681) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:24,950::__init__::232::IOProcess::(_processLogs) (7681) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:24,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:24,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:24,952::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:24,952::__init__::232::IOProcess::(_processLogs) (7682) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:24,952::__init__::232::IOProcess::(_processLogs) (7682) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:24,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:24,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:24,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:24,955::__init__::232::IOProcess::(_processLogs) (7683) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:24,955::__init__::232::IOProcess::(_processLogs) (7683) Queuing response >Thread-122::DEBUG::2014-08-11 15:46:25,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff2b2ee1-d2e6-465c-b726-838c35907b2c`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:46:25,649::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:46:25,650::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:25,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:25,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:25,651::__init__::232::IOProcess::(_processLogs) (7684) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:46:25,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7684) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:25,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:25,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:46:25,654::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:46:25,655::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:25,656::__init__::232::IOProcess::(_processLogs) (7685) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:46:25,656::task::1191::Storage.TaskManager.Task::(prepare) Task=`ff2b2ee1-d2e6-465c-b726-838c35907b2c`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:46:25,657::__init__::232::IOProcess::(_processLogs) (7685) Queuing response >Thread-122::DEBUG::2014-08-11 15:46:25,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`ff2b2ee1-d2e6-465c-b726-838c35907b2c`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:46:25,658::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:46:25,658::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:46:25,659::task::993::Storage.TaskManager.Task::(_decref) Task=`ff2b2ee1-d2e6-465c-b726-838c35907b2c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:46:26,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:26,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136322 s, 75.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:26,988::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:26,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:27,074::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:28,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:28,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132895 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:30,712::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:30,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134507 s, 76.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:32,039::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:46:32,042::task::595::Storage.TaskManager.Task::(_updateState) Task=`044b3896-9deb-4ef0-8115-7b04b4ebbc1e`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:32,074::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:32,077::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:32,079::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:32,081::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:46:32,089::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:32,091::task::1191::Storage.TaskManager.Task::(prepare) Task=`044b3896-9deb-4ef0-8115-7b04b4ebbc1e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:32,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`044b3896-9deb-4ef0-8115-7b04b4ebbc1e`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:32,093::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:32,094::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:32,095::task::993::Storage.TaskManager.Task::(_decref) Task=`044b3896-9deb-4ef0-8115-7b04b4ebbc1e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:46:32,109::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:46:32,113::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf0d8251-7e96-4a9a-86fd-49137b6c2721`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:46:32,114::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:46:32,119::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`561e78f6-d805-42af-8f58-6461aba7eee9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:46:32,121::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:46:32,124::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:46:32,125::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`561e78f6-d805-42af-8f58-6461aba7eee9`::Granted request >Thread-34::DEBUG::2014-08-11 15:46:32,129::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bf0d8251-7e96-4a9a-86fd-49137b6c2721`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:46:32,131::task::993::Storage.TaskManager.Task::(_decref) Task=`bf0d8251-7e96-4a9a-86fd-49137b6c2721`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:46:32,181::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:32,182::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf0d8251-7e96-4a9a-86fd-49137b6c2721`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:32,183::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf0d8251-7e96-4a9a-86fd-49137b6c2721`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:46:32,184::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:46:32,186::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:46:32,187::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:46:32,189::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:46:32,190::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:46:32,191::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:46:32,193::task::993::Storage.TaskManager.Task::(_decref) Task=`bf0d8251-7e96-4a9a-86fd-49137b6c2721`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:46:32,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:32,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0267821 s, 38.2 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:32,910::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5e1e7d0-f7b1-4c78-add5-5d749b88dae9`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:32,912::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:46:32,914::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000495739', 'lastCheck': '8.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:32,915::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5e1e7d0-f7b1-4c78-add5-5d749b88dae9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000495739', 'lastCheck': '8.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:32,917::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5e1e7d0-f7b1-4c78-add5-5d749b88dae9`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:32,918::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:32,919::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:32,920::task::993::Storage.TaskManager.Task::(_decref) Task=`b5e1e7d0-f7b1-4c78-add5-5d749b88dae9`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:46:32,975::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:46:34,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:34,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:34,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:34,956::__init__::232::IOProcess::(_processLogs) (7686) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:34,956::__init__::232::IOProcess::(_processLogs) (7686) Queuing response >Thread-16::DEBUG::2014-08-11 15:46:34,957::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:34,981::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-16::DEBUG::2014-08-11 15:46:35,006::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00075165 s, 605 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:35,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:35,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:35,012::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:35,012::__init__::232::IOProcess::(_processLogs) (7687) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:35,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7687) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:35,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:35,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:35,025::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:35,026::__init__::232::IOProcess::(_processLogs) (7688) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:35,028::__init__::232::IOProcess::(_processLogs) (7688) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:35,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:35,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:35,033::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:35,034::__init__::232::IOProcess::(_processLogs) (7689) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:35,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7689) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:35,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:35,041::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 15:46:35,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.032326 s, 31.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:35,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:35,044::__init__::232::IOProcess::(_processLogs) (7690) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:35,046::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7690) Queuing response >Dummy-33::DEBUG::2014-08-11 15:46:37,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:37,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:37,090::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:37,092::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:37,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013264 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:39,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:39,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132686 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:41,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:41,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132732 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:42,097::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:42,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:42,174::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:46:42,208::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-128::DEBUG::2014-08-11 15:46:42,209::task::595::Storage.TaskManager.Task::(_updateState) Task=`37b04d1a-430e-45b3-a3d3-0a1a9d517990`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:42,210::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:46:42,223::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:42,223::task::1191::Storage.TaskManager.Task::(prepare) Task=`37b04d1a-430e-45b3-a3d3-0a1a9d517990`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:42,224::task::595::Storage.TaskManager.Task::(_updateState) Task=`37b04d1a-430e-45b3-a3d3-0a1a9d517990`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:42,224::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:42,224::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:42,224::task::993::Storage.TaskManager.Task::(_decref) Task=`37b04d1a-430e-45b3-a3d3-0a1a9d517990`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:46:42,235::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c2417b4] >Thread-34::DEBUG::2014-08-11 15:46:42,235::task::595::Storage.TaskManager.Task::(_updateState) Task=`031e968e-a0c5-43ab-ae48-12e663fe2d3b`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:46:42,236::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:46:42,236::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1f81c553-216b-4a42-bcc8-4c3782c6ff17`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:46:42,237::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:46:42,237::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:46:42,237::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`1f81c553-216b-4a42-bcc8-4c3782c6ff17`::Granted request >Thread-34::DEBUG::2014-08-11 15:46:42,238::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`031e968e-a0c5-43ab-ae48-12e663fe2d3b`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:46:42,238::task::993::Storage.TaskManager.Task::(_decref) Task=`031e968e-a0c5-43ab-ae48-12e663fe2d3b`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:46:42,258::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:42,258::task::1191::Storage.TaskManager.Task::(prepare) Task=`031e968e-a0c5-43ab-ae48-12e663fe2d3b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:42,258::task::595::Storage.TaskManager.Task::(_updateState) Task=`031e968e-a0c5-43ab-ae48-12e663fe2d3b`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:46:42,259::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:46:42,259::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:46:42,259::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:46:42,260::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:46:42,260::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:46:42,260::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:46:42,260::task::993::Storage.TaskManager.Task::(_decref) Task=`031e968e-a0c5-43ab-ae48-12e663fe2d3b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:46:43,236::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:43,270::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132216 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:45,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:45,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:45,049::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:45,049::__init__::232::IOProcess::(_processLogs) (7691) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:45,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7691) Queuing response >Thread-16::DEBUG::2014-08-11 15:46:45,050::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:46:45,074::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000853446 s, 533 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:45,076::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:45,076::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:45,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:45,077::__init__::232::IOProcess::(_processLogs) (7692) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:45,077::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7692) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:45,078::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:45,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:45,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:45,079::__init__::232::IOProcess::(_processLogs) (7693) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:45,080::__init__::232::IOProcess::(_processLogs) (7693) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:45,080::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:45,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:45,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:45,081::__init__::232::IOProcess::(_processLogs) (7694) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:45,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7694) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:45,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:45,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:45,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:45,083::__init__::232::IOProcess::(_processLogs) (7695) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:45,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7695) Queuing response >Dummy-33::DEBUG::2014-08-11 15:46:45,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:45,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132241 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:47,181::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:47,182::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:47,274::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:47,349::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:47,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134292 s, 76.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:48,086::task::595::Storage.TaskManager.Task::(_updateState) Task=`79499db1-a953-42a8-9392-0b64dfd34ca5`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:48,087::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:46:48,087::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000853446', 'lastCheck': '3.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:48,087::task::1191::Storage.TaskManager.Task::(prepare) Task=`79499db1-a953-42a8-9392-0b64dfd34ca5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000853446', 'lastCheck': '3.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:46:48,088::task::595::Storage.TaskManager.Task::(_updateState) Task=`79499db1-a953-42a8-9392-0b64dfd34ca5`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:48,088::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:48,088::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:48,088::task::993::Storage.TaskManager.Task::(_decref) Task=`79499db1-a953-42a8-9392-0b64dfd34ca5`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:46:48,107::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:49,406::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:49,439::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133324 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:51,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:51,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136327 s, 75.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:46:52,284::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:52,284::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:46:52,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`380dec96-b0d1-4bdb-a23c-db70ab43384a`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:46:52,288::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:52,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:52,291::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:46:52,302::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:52,303::task::1191::Storage.TaskManager.Task::(prepare) Task=`380dec96-b0d1-4bdb-a23c-db70ab43384a`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:46:52,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`380dec96-b0d1-4bdb-a23c-db70ab43384a`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:46:52,303::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:46:52,303::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:46:52,304::task::993::Storage.TaskManager.Task::(_decref) Task=`380dec96-b0d1-4bdb-a23c-db70ab43384a`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:46:52,319::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-34::DEBUG::2014-08-11 15:46:52,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d2d866a-6ec9-4b11-af6a-b29043cbe496`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:46:52,320::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:46:52,321::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`afe2177d-97de-4892-b421-4a9ff1259ad0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:46:52,321::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:46:52,322::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:46:52,322::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`afe2177d-97de-4892-b421-4a9ff1259ad0`::Granted request >Thread-34::DEBUG::2014-08-11 15:46:52,322::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2d2d866a-6ec9-4b11-af6a-b29043cbe496`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:46:52,323::task::993::Storage.TaskManager.Task::(_decref) Task=`2d2d866a-6ec9-4b11-af6a-b29043cbe496`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:46:52,342::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:52,343::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d2d866a-6ec9-4b11-af6a-b29043cbe496`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:46:52,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d2d866a-6ec9-4b11-af6a-b29043cbe496`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:46:52,343::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:46:52,343::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:46:52,344::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:46:52,344::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:46:52,344::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:46:52,345::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:46:52,345::task::993::Storage.TaskManager.Task::(_decref) Task=`2d2d866a-6ec9-4b11-af6a-b29043cbe496`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:46:53,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:53,551::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131494 s, 77.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:55,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:55,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:55,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:55,087::__init__::232::IOProcess::(_processLogs) (7696) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:55,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7696) Queuing response >Thread-16::DEBUG::2014-08-11 15:46:55,088::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:46:55,116::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00603921 s, 75.3 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:46:55,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:55,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:55,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:55,118::__init__::232::IOProcess::(_processLogs) (7697) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:46:55,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7697) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:55,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:55,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:55,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:55,121::__init__::232::IOProcess::(_processLogs) (7698) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:55,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7698) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:55,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:55,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:55,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:55,122::__init__::232::IOProcess::(_processLogs) (7699) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:55,123::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7699) Queuing response >Thread-14::DEBUG::2014-08-11 15:46:55,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:46:55,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:46:55,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:46:55,125::__init__::232::IOProcess::(_processLogs) (7700) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:46:55,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7700) Queuing response >Dummy-33::DEBUG::2014-08-11 15:46:55,573::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:55,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160625 s, 63.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:57,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:57,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:46:57,296::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:46:57,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:57,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132085 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:46:59,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:46:59,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131209 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:01,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:01,799::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133798 s, 76.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:02,304::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:02,311::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:47:02,359::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-128::DEBUG::2014-08-11 15:47:02,377::task::595::Storage.TaskManager.Task::(_updateState) Task=`04ee303a-cc63-4125-afed-99d089a285b2`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:02,379::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:02,381::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:47:02,394::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:02,395::task::1191::Storage.TaskManager.Task::(prepare) Task=`04ee303a-cc63-4125-afed-99d089a285b2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:02,396::task::595::Storage.TaskManager.Task::(_updateState) Task=`04ee303a-cc63-4125-afed-99d089a285b2`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:02,397::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:02,398::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:02,399::task::993::Storage.TaskManager.Task::(_decref) Task=`04ee303a-cc63-4125-afed-99d089a285b2`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:47:02,413::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [1c294e69] >Thread-34::DEBUG::2014-08-11 15:47:02,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5589f07-2e4a-428f-9123-138f86cf6345`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:47:02,419::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:47:02,423::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`81b27dbb-f6b9-45a5-b9ec-f551e3491d97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:47:02,424::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:47:02,428::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:47:02,430::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`81b27dbb-f6b9-45a5-b9ec-f551e3491d97`::Granted request >Thread-34::DEBUG::2014-08-11 15:47:02,435::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b5589f07-2e4a-428f-9123-138f86cf6345`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:47:02,436::task::993::Storage.TaskManager.Task::(_decref) Task=`b5589f07-2e4a-428f-9123-138f86cf6345`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:47:02,491::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:02,493::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5589f07-2e4a-428f-9123-138f86cf6345`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:02,493::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5589f07-2e4a-428f-9123-138f86cf6345`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:47:02,494::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:47:02,495::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:47:02,497::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:47:02,499::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:47:02,500::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:47:02,501::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:47:02,503::task::993::Storage.TaskManager.Task::(_decref) Task=`b5589f07-2e4a-428f-9123-138f86cf6345`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:47:03,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`afa601a7-ce2d-4cc6-b450-55487ebeb84b`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:03,203::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:47:03,203::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00603921', 'lastCheck': '8.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:03,204::task::1191::Storage.TaskManager.Task::(prepare) Task=`afa601a7-ce2d-4cc6-b450-55487ebeb84b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00603921', 'lastCheck': '8.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:03,204::task::595::Storage.TaskManager.Task::(_updateState) Task=`afa601a7-ce2d-4cc6-b450-55487ebeb84b`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:03,204::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:03,204::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:03,204::task::993::Storage.TaskManager.Task::(_decref) Task=`afa601a7-ce2d-4cc6-b450-55487ebeb84b`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:47:03,222::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:03,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:03,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134136 s, 76.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:05,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:05,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:05,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:05,128::__init__::232::IOProcess::(_processLogs) (7701) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:05,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7701) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:05,129::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:47:05,161::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00706005 s, 64.4 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:05,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:05,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:05,163::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:05,163::__init__::232::IOProcess::(_processLogs) (7702) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:05,164::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7702) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:05,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:05,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:05,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:05,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7703) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:05,166::__init__::232::IOProcess::(_processLogs) (7703) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:05,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:05,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:05,168::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:05,168::__init__::232::IOProcess::(_processLogs) (7704) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:05,168::__init__::232::IOProcess::(_processLogs) (7704) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:05,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:05,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:05,170::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:05,171::__init__::232::IOProcess::(_processLogs) (7705) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:05,171::__init__::232::IOProcess::(_processLogs) (7705) Queuing response >Dummy-33::DEBUG::2014-08-11 15:47:05,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:05,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131947 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:07,389::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:07,391::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:07,476::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:07,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:07,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215475 s, 47.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:09,998::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:10,032::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131428 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:12,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:12,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132099 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:12,483::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:12,485::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:47:12,519::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:47:12,577::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5d3d5cc-b64e-47bc-a44d-f1fee9780321`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:12,579::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:12,579::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:47:12,593::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:12,593::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5d3d5cc-b64e-47bc-a44d-f1fee9780321`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:12,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5d3d5cc-b64e-47bc-a44d-f1fee9780321`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:12,594::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:12,594::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:12,594::task::993::Storage.TaskManager.Task::(_decref) Task=`d5d3d5cc-b64e-47bc-a44d-f1fee9780321`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:47:12,604::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:47:12,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bec9c87-ae7e-4153-8965-be0d5acf40c8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:47:12,605::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:47:12,606::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`209ef516-ccfc-433a-8258-aa81fe4e9742`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:47:12,606::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:47:12,607::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:47:12,607::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`209ef516-ccfc-433a-8258-aa81fe4e9742`::Granted request >Thread-34::DEBUG::2014-08-11 15:47:12,607::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5bec9c87-ae7e-4153-8965-be0d5acf40c8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:47:12,608::task::993::Storage.TaskManager.Task::(_decref) Task=`5bec9c87-ae7e-4153-8965-be0d5acf40c8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:47:12,627::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:12,628::task::1191::Storage.TaskManager.Task::(prepare) Task=`5bec9c87-ae7e-4153-8965-be0d5acf40c8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:12,628::task::595::Storage.TaskManager.Task::(_updateState) Task=`5bec9c87-ae7e-4153-8965-be0d5acf40c8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:47:12,628::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:47:12,629::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:47:12,629::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:47:12,629::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:47:12,629::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:47:12,630::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:47:12,630::task::993::Storage.TaskManager.Task::(_decref) Task=`5bec9c87-ae7e-4153-8965-be0d5acf40c8`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:47:14,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:14,144::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133031 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:15,171::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:15,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:15,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:15,172::__init__::232::IOProcess::(_processLogs) (7706) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:15,172::__init__::232::IOProcess::(_processLogs) (7706) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:15,173::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:47:15,204::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00424337 s, 107 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:15,205::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:15,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:15,206::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:15,206::__init__::232::IOProcess::(_processLogs) (7707) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:15,207::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7707) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:15,208::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:15,209::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:15,209::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:15,209::__init__::232::IOProcess::(_processLogs) (7708) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:15,210::__init__::232::IOProcess::(_processLogs) (7708) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:15,210::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:15,211::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:15,211::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:15,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7709) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:15,212::__init__::232::IOProcess::(_processLogs) (7709) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:15,213::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:15,213::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:15,214::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:15,214::__init__::232::IOProcess::(_processLogs) (7710) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:15,215::__init__::232::IOProcess::(_processLogs) (7710) Queuing response >Dummy-33::DEBUG::2014-08-11 15:47:16,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:16,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131736 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:17,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:17,588::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:17,675::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:18,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:18,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131727 s, 77.7 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:47:18,312::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dc507f5-c3da-44b9-9b66-267ff7f82e55`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:18,313::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:47:18,313::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00424337', 'lastCheck': '3.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:18,313::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dc507f5-c3da-44b9-9b66-267ff7f82e55`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00424337', 'lastCheck': '3.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:18,314::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dc507f5-c3da-44b9-9b66-267ff7f82e55`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:18,314::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:18,314::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:18,314::task::993::Storage.TaskManager.Task::(_decref) Task=`9dc507f5-c3da-44b9-9b66-267ff7f82e55`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:47:18,339::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:20,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:20,311::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133264 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:22,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:22,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133014 s, 77.0 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:47:22,641::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:47:22,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf30be0e-61b0-4eba-8bb2-cb13fd128313`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:22,674::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:22,682::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:22,684::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:22,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:47:22,687::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:22,688::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf30be0e-61b0-4eba-8bb2-cb13fd128313`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:22,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf30be0e-61b0-4eba-8bb2-cb13fd128313`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:22,688::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:22,688::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:22,688::task::993::Storage.TaskManager.Task::(_decref) Task=`cf30be0e-61b0-4eba-8bb2-cb13fd128313`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:47:22,700::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:47:22,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a219bf9-8998-4f0c-9ee0-ef15485b5d3e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:47:22,701::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:47:22,701::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9b1545cd-9869-41b8-9d9d-1fbfc78228cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:47:22,702::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:47:22,702::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:47:22,702::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9b1545cd-9869-41b8-9d9d-1fbfc78228cf`::Granted request >Thread-34::DEBUG::2014-08-11 15:47:22,703::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9a219bf9-8998-4f0c-9ee0-ef15485b5d3e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:47:22,703::task::993::Storage.TaskManager.Task::(_decref) Task=`9a219bf9-8998-4f0c-9ee0-ef15485b5d3e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:47:22,723::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:22,723::task::1191::Storage.TaskManager.Task::(prepare) Task=`9a219bf9-8998-4f0c-9ee0-ef15485b5d3e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:22,723::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a219bf9-8998-4f0c-9ee0-ef15485b5d3e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:47:22,724::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:47:22,724::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:47:22,724::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:47:22,725::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:47:22,725::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:47:22,725::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:47:22,725::task::993::Storage.TaskManager.Task::(_decref) Task=`9a219bf9-8998-4f0c-9ee0-ef15485b5d3e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:47:24,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:24,427::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0171127 s, 59.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:25,215::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:25,215::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:25,216::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:25,216::__init__::232::IOProcess::(_processLogs) (7711) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:25,216::__init__::232::IOProcess::(_processLogs) (7711) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:25,217::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:47:25,240::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00214782 s, 212 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:25,242::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:25,243::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:25,243::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:25,243::__init__::232::IOProcess::(_processLogs) (7712) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:25,244::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7712) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:25,245::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:25,246::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:25,246::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:25,246::__init__::232::IOProcess::(_processLogs) (7713) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:25,246::__init__::232::IOProcess::(_processLogs) (7713) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:25,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:25,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:25,248::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:25,248::__init__::232::IOProcess::(_processLogs) (7714) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:25,249::__init__::232::IOProcess::(_processLogs) (7714) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:25,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:25,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:25,250::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:25,251::__init__::232::IOProcess::(_processLogs) (7715) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:25,252::__init__::232::IOProcess::(_processLogs) (7715) Queuing response >Thread-122::DEBUG::2014-08-11 15:47:25,763::task::595::Storage.TaskManager.Task::(_updateState) Task=`29444789-46b2-40cb-8020-5973e62d2862`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:47:25,763::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:47:25,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:25,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:25,765::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:25,765::__init__::232::IOProcess::(_processLogs) (7716) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:47:25,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7716) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:25,767::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:25,767::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:47:25,767::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:47:25,768::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:47:25,769::task::1191::Storage.TaskManager.Task::(prepare) Task=`29444789-46b2-40cb-8020-5973e62d2862`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:47:25,770::__init__::232::IOProcess::(_processLogs) (7717) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:47:25,770::task::595::Storage.TaskManager.Task::(_updateState) Task=`29444789-46b2-40cb-8020-5973e62d2862`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:47:25,771::__init__::232::IOProcess::(_processLogs) (7717) Queuing response >Thread-122::DEBUG::2014-08-11 15:47:25,772::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:47:25,772::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:47:25,773::task::993::Storage.TaskManager.Task::(_decref) Task=`29444789-46b2-40cb-8020-5973e62d2862`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:47:26,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:26,482::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132816 s, 77.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:27,689::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:27,690::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:27,775::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:28,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:28,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133035 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:30,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:30,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135344 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:32,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:32,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250846 s, 40.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:47:32,739::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:47:32,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`eebf5916-a8e8-4b39-a942-0634e7c67a4f`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:32,776::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:32,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:32,788::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:32,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:47:32,803::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:32,807::task::1191::Storage.TaskManager.Task::(prepare) Task=`eebf5916-a8e8-4b39-a942-0634e7c67a4f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:32,809::task::595::Storage.TaskManager.Task::(_updateState) Task=`eebf5916-a8e8-4b39-a942-0634e7c67a4f`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:32,810::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:32,812::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:32,814::task::993::Storage.TaskManager.Task::(_decref) Task=`eebf5916-a8e8-4b39-a942-0634e7c67a4f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:47:32,829::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:47:32,832::task::595::Storage.TaskManager.Task::(_updateState) Task=`73e1e8b2-8971-4c0d-807f-f847c51d2028`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:47:32,834::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:47:32,837::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3608d916-7f77-4745-8f76-810ceeb31085`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:47:32,838::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:47:32,840::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:47:32,842::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3608d916-7f77-4745-8f76-810ceeb31085`::Granted request >Thread-34::DEBUG::2014-08-11 15:47:32,844::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`73e1e8b2-8971-4c0d-807f-f847c51d2028`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:47:32,846::task::993::Storage.TaskManager.Task::(_decref) Task=`73e1e8b2-8971-4c0d-807f-f847c51d2028`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:47:32,901::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:32,903::task::1191::Storage.TaskManager.Task::(prepare) Task=`73e1e8b2-8971-4c0d-807f-f847c51d2028`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:32,904::task::595::Storage.TaskManager.Task::(_updateState) Task=`73e1e8b2-8971-4c0d-807f-f847c51d2028`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:47:32,905::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:47:32,906::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:47:32,908::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:47:32,910::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:47:32,911::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:47:32,912::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:47:32,914::task::993::Storage.TaskManager.Task::(_decref) Task=`73e1e8b2-8971-4c0d-807f-f847c51d2028`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:47:33,428::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b1be265-762d-49e1-b893-3331fdca19cd`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:33,428::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:47:33,428::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00214782', 'lastCheck': '8.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:33,429::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b1be265-762d-49e1-b893-3331fdca19cd`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00214782', 'lastCheck': '8.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:33,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b1be265-762d-49e1-b893-3331fdca19cd`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:33,429::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:33,429::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:33,429::task::993::Storage.TaskManager.Task::(_decref) Task=`1b1be265-762d-49e1-b893-3331fdca19cd`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:47:33,450::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:34,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:34,964::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131658 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:35,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:35,253::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:35,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:35,253::__init__::232::IOProcess::(_processLogs) (7718) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:35,253::__init__::232::IOProcess::(_processLogs) (7718) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:35,254::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:47:35,275::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000551965 s, 824 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:35,277::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:35,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:35,278::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:35,279::__init__::232::IOProcess::(_processLogs) (7719) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:35,279::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7719) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:35,281::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:35,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:35,282::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:35,283::__init__::232::IOProcess::(_processLogs) (7720) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:35,283::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7720) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:35,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:35,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:35,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:35,286::__init__::232::IOProcess::(_processLogs) (7721) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:35,286::__init__::232::IOProcess::(_processLogs) (7721) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:35,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:35,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:35,288::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:35,288::__init__::232::IOProcess::(_processLogs) (7722) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:35,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7722) Queuing response >Dummy-33::DEBUG::2014-08-11 15:47:36,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:37,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013328 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:37,805::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:37,876::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:37,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:39,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:39,077::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133241 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:41,099::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:41,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132615 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:42,880::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:42,881::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:47:42,930::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:47:42,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`34f05771-4c95-468e-99fb-a8be575b3530`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:42,980::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:42,980::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:47:42,995::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:42,995::task::1191::Storage.TaskManager.Task::(prepare) Task=`34f05771-4c95-468e-99fb-a8be575b3530`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:42,995::task::595::Storage.TaskManager.Task::(_updateState) Task=`34f05771-4c95-468e-99fb-a8be575b3530`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:42,995::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:42,996::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:42,996::task::993::Storage.TaskManager.Task::(_decref) Task=`34f05771-4c95-468e-99fb-a8be575b3530`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:47:43,007::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:47:43,008::task::595::Storage.TaskManager.Task::(_updateState) Task=`f96642c3-3925-42db-a70a-43d5fe5e9086`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:47:43,008::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:47:43,009::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`311223d7-ced5-4065-88e2-190aab3ebfc0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:47:43,009::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:47:43,010::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:47:43,010::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`311223d7-ced5-4065-88e2-190aab3ebfc0`::Granted request >Thread-34::DEBUG::2014-08-11 15:47:43,010::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f96642c3-3925-42db-a70a-43d5fe5e9086`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:47:43,011::task::993::Storage.TaskManager.Task::(_decref) Task=`f96642c3-3925-42db-a70a-43d5fe5e9086`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:47:43,030::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:43,031::task::1191::Storage.TaskManager.Task::(prepare) Task=`f96642c3-3925-42db-a70a-43d5fe5e9086`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:43,031::task::595::Storage.TaskManager.Task::(_updateState) Task=`f96642c3-3925-42db-a70a-43d5fe5e9086`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:47:43,031::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:47:43,032::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:47:43,032::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:47:43,032::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:47:43,032::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:47:43,033::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:47:43,033::task::993::Storage.TaskManager.Task::(_decref) Task=`f96642c3-3925-42db-a70a-43d5fe5e9086`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:47:43,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:43,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131607 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:45,212::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:45,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133001 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:45,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:45,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:45,291::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:45,291::__init__::232::IOProcess::(_processLogs) (7723) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:45,291::__init__::232::IOProcess::(_processLogs) (7723) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:45,292::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:47:45,312::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000521703 s, 872 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:45,314::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:45,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:45,316::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:45,317::__init__::232::IOProcess::(_processLogs) (7724) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:45,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7724) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:45,322::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:45,323::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:45,324::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:45,325::__init__::232::IOProcess::(_processLogs) (7725) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:45,326::__init__::232::IOProcess::(_processLogs) (7725) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:45,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:45,328::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:45,328::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:45,329::__init__::232::IOProcess::(_processLogs) (7726) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:45,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7726) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:45,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:45,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:45,331::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:45,332::__init__::232::IOProcess::(_processLogs) (7727) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:45,332::__init__::232::IOProcess::(_processLogs) (7727) Queuing response >Dummy-33::DEBUG::2014-08-11 15:47:47,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:47,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132945 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:47,988::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:47,989::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:48,076::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:47:48,572::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e25bda7-cb67-4399-9ad8-c6a071377ed0`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:48,573::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:47:48,573::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000521703', 'lastCheck': '3.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:48,574::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e25bda7-cb67-4399-9ad8-c6a071377ed0`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000521703', 'lastCheck': '3.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:47:48,574::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e25bda7-cb67-4399-9ad8-c6a071377ed0`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:48,574::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:48,574::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:48,574::task::993::Storage.TaskManager.Task::(_decref) Task=`7e25bda7-cb67-4399-9ad8-c6a071377ed0`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:47:48,593::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:49,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:49,374::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0250265 s, 40.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:51,565::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:51,599::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013347 s, 76.7 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:47:53,045::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:47:53,046::task::595::Storage.TaskManager.Task::(_updateState) Task=`294404da-5b78-4305-a250-3007df1a2b9c`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:47:53,075::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:53,079::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:53,080::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:53,081::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:47:53,088::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:53,089::task::1191::Storage.TaskManager.Task::(prepare) Task=`294404da-5b78-4305-a250-3007df1a2b9c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:47:53,089::task::595::Storage.TaskManager.Task::(_updateState) Task=`294404da-5b78-4305-a250-3007df1a2b9c`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:47:53,089::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:47:53,089::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:47:53,090::task::993::Storage.TaskManager.Task::(_decref) Task=`294404da-5b78-4305-a250-3007df1a2b9c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:47:53,101::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:47:53,101::task::595::Storage.TaskManager.Task::(_updateState) Task=`6aa4a3c8-0e35-4768-9696-47528d573221`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:47:53,101::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:47:53,102::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`734ad1c4-9d9a-40df-8c67-d8122662e9f5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:47:53,102::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:47:53,103::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:47:53,103::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`734ad1c4-9d9a-40df-8c67-d8122662e9f5`::Granted request >Thread-34::DEBUG::2014-08-11 15:47:53,104::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6aa4a3c8-0e35-4768-9696-47528d573221`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:47:53,104::task::993::Storage.TaskManager.Task::(_decref) Task=`6aa4a3c8-0e35-4768-9696-47528d573221`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:47:53,124::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:53,124::task::1191::Storage.TaskManager.Task::(prepare) Task=`6aa4a3c8-0e35-4768-9696-47528d573221`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952084480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:47:53,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`6aa4a3c8-0e35-4768-9696-47528d573221`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:47:53,124::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:47:53,125::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:47:53,125::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:47:53,125::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:47:53,125::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:47:53,126::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:47:53,126::task::993::Storage.TaskManager.Task::(_decref) Task=`6aa4a3c8-0e35-4768-9696-47528d573221`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:47:53,622::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:53,655::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013179 s, 77.7 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:47:55,334::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:47:55,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,336::__init__::232::IOProcess::(_processLogs) (7728) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:47:55,337::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7728) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:55,347::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:47:55,348::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,348::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,348::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,349::__init__::232::IOProcess::(_processLogs) (7729) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:47:55,349::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7729) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:55,350::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:47:55,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,351::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,351::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,351::__init__::232::IOProcess::(_processLogs) (7730) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 15:47:55,352::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:47:55,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7730) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:55,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,355::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,356::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,356::__init__::232::IOProcess::(_processLogs) (7731) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:55,356::__init__::232::IOProcess::(_processLogs) (7731) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:55,357::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,357::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,357::__init__::232::IOProcess::(_processLogs) (7732) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:47:55,358::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-14::DEBUG::2014-08-11 15:47:55,358::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7732) Queuing response >Thread-16::INFO::2014-08-11 15:47:55,359::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:47:55,360::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:47:55,360::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,361::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,361::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,361::__init__::232::IOProcess::(_processLogs) (7733) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:55,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7733) Queuing response >Thread-16::DEBUG::2014-08-11 15:47:55,363::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:47:55,383::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420924 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:47:55,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,385::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,385::__init__::232::IOProcess::(_processLogs) (7734) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:47:55,386::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7734) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:55,387::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,389::__init__::232::IOProcess::(_processLogs) (7735) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:55,389::__init__::232::IOProcess::(_processLogs) (7735) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:55,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,391::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,391::__init__::232::IOProcess::(_processLogs) (7736) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:55,391::__init__::232::IOProcess::(_processLogs) (7736) Queuing response >Thread-14::DEBUG::2014-08-11 15:47:55,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:47:55,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:47:55,393::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:47:55,394::__init__::232::IOProcess::(_processLogs) (7737) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:47:55,394::__init__::232::IOProcess::(_processLogs) (7737) Queuing response >Dummy-33::DEBUG::2014-08-11 15:47:55,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:55,711::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131529 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:47:57,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:57,768::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131633 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:58,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:58,088::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:47:58,177::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:47:59,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:47:59,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130766 s, 78.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:01,847::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:01,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134205 s, 76.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:48:03,138::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:48:03,139::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a41b365-6190-471e-a343-e8e6ef61861c`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:03,139::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:48:03,152::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:03,152::task::1191::Storage.TaskManager.Task::(prepare) Task=`0a41b365-6190-471e-a343-e8e6ef61861c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:03,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`0a41b365-6190-471e-a343-e8e6ef61861c`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:03,153::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:03,153::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:03,153::task::993::Storage.TaskManager.Task::(_decref) Task=`0a41b365-6190-471e-a343-e8e6ef61861c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:48:03,164::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:48:03,164::task::595::Storage.TaskManager.Task::(_updateState) Task=`be78cf02-1312-4a07-81e4-28f2a84e4ab7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:48:03,165::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:48:03,165::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4b8fd32f-b42d-4d46-bed5-a97074c1a139`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:48:03,166::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:48:03,166::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:48:03,168::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4b8fd32f-b42d-4d46-bed5-a97074c1a139`::Granted request >Thread-34::DEBUG::2014-08-11 15:48:03,169::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`be78cf02-1312-4a07-81e4-28f2a84e4ab7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:48:03,169::task::993::Storage.TaskManager.Task::(_decref) Task=`be78cf02-1312-4a07-81e4-28f2a84e4ab7`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:03,184::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:03,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:03,187::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:48:03,189::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952018944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:03,189::task::1191::Storage.TaskManager.Task::(prepare) Task=`be78cf02-1312-4a07-81e4-28f2a84e4ab7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952018944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:03,189::task::595::Storage.TaskManager.Task::(_updateState) Task=`be78cf02-1312-4a07-81e4-28f2a84e4ab7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:48:03,190::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:48:03,190::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:48:03,190::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:48:03,191::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:48:03,191::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:48:03,191::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:48:03,191::task::993::Storage.TaskManager.Task::(_decref) Task=`be78cf02-1312-4a07-81e4-28f2a84e4ab7`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:48:03,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`922d8630-e40d-4b0e-8ebc-6e70a90c06b3`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:03,683::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:48:03,683::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420924', 'lastCheck': '8.3', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:03,684::task::1191::Storage.TaskManager.Task::(prepare) Task=`922d8630-e40d-4b0e-8ebc-6e70a90c06b3`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420924', 'lastCheck': '8.3', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:03,684::task::595::Storage.TaskManager.Task::(_updateState) Task=`922d8630-e40d-4b0e-8ebc-6e70a90c06b3`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:03,684::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:03,684::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:03,684::task::993::Storage.TaskManager.Task::(_decref) Task=`922d8630-e40d-4b0e-8ebc-6e70a90c06b3`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:48:03,703::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:03,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:03,939::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132197 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:05,394::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:05,395::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:05,395::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:05,395::__init__::232::IOProcess::(_processLogs) (7738) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:05,396::__init__::232::IOProcess::(_processLogs) (7738) Queuing response >Thread-16::DEBUG::2014-08-11 15:48:05,396::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:48:05,416::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417545 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:05,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:05,418::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:05,418::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:05,418::__init__::232::IOProcess::(_processLogs) (7739) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:05,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7739) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:05,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:05,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:05,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:05,422::__init__::232::IOProcess::(_processLogs) (7740) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:05,422::__init__::232::IOProcess::(_processLogs) (7740) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:05,422::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:05,423::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:05,423::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:05,424::__init__::232::IOProcess::(_processLogs) (7741) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:05,424::__init__::232::IOProcess::(_processLogs) (7741) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:05,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:05,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:05,426::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:05,426::__init__::232::IOProcess::(_processLogs) (7742) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:05,427::__init__::232::IOProcess::(_processLogs) (7742) Queuing response >Dummy-33::DEBUG::2014-08-11 15:48:05,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:05,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013198 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:08,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:08,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132283 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:08,192::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:08,193::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:08,277::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:10,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:10,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132306 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:12,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:12,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132428 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:48:13,202::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:48:13,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`725282f5-61ac-4356-9f26-b77f3b8e3824`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:13,276::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:13,284::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:13,286::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:13,287::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:48:13,289::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:13,290::task::1191::Storage.TaskManager.Task::(prepare) Task=`725282f5-61ac-4356-9f26-b77f3b8e3824`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:13,290::task::595::Storage.TaskManager.Task::(_updateState) Task=`725282f5-61ac-4356-9f26-b77f3b8e3824`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:13,290::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:13,290::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:13,290::task::993::Storage.TaskManager.Task::(_decref) Task=`725282f5-61ac-4356-9f26-b77f3b8e3824`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:48:13,300::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:48:13,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fed0476-8b57-428f-9871-8137536cf26c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:48:13,301::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:48:13,302::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ac497c7c-fc37-43b6-85fd-643c1dd38b35`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:48:13,302::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:48:13,302::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:48:13,303::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ac497c7c-fc37-43b6-85fd-643c1dd38b35`::Granted request >Thread-34::DEBUG::2014-08-11 15:48:13,303::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6fed0476-8b57-428f-9871-8137536cf26c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:48:13,303::task::993::Storage.TaskManager.Task::(_decref) Task=`6fed0476-8b57-428f-9871-8137536cf26c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:48:13,323::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952018944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:13,324::task::1191::Storage.TaskManager.Task::(prepare) Task=`6fed0476-8b57-428f-9871-8137536cf26c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952018944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:13,324::task::595::Storage.TaskManager.Task::(_updateState) Task=`6fed0476-8b57-428f-9871-8137536cf26c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:48:13,324::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:48:13,324::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:48:13,325::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:48:13,325::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:48:13,325::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:48:13,326::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:48:13,326::task::993::Storage.TaskManager.Task::(_decref) Task=`6fed0476-8b57-428f-9871-8137536cf26c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:48:14,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:14,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132112 s, 77.5 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:15,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:15,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:15,428::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:15,428::__init__::232::IOProcess::(_processLogs) (7743) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:15,428::__init__::232::IOProcess::(_processLogs) (7743) Queuing response >Thread-16::DEBUG::2014-08-11 15:48:15,429::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:48:15,449::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425281 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:15,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:15,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:15,451::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:15,451::__init__::232::IOProcess::(_processLogs) (7744) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:15,451::__init__::232::IOProcess::(_processLogs) (7744) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:15,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:15,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:15,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:15,454::__init__::232::IOProcess::(_processLogs) (7745) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:15,454::__init__::232::IOProcess::(_processLogs) (7745) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:15,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:15,455::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:15,456::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:15,456::__init__::232::IOProcess::(_processLogs) (7746) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:15,456::__init__::232::IOProcess::(_processLogs) (7746) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:15,456::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:15,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:15,458::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:15,458::__init__::232::IOProcess::(_processLogs) (7747) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:15,459::__init__::232::IOProcess::(_processLogs) (7747) Queuing response >Dummy-33::DEBUG::2014-08-11 15:48:16,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:16,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132416 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:18,293::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:18,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:18,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:18,398::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:18,419::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0220712 s, 46.4 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:48:18,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`3bb29612-ed23-4888-bc20-dcb24577ca4a`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:18,800::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:48:18,801::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425281', 'lastCheck': '3.3', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:18,801::task::1191::Storage.TaskManager.Task::(prepare) Task=`3bb29612-ed23-4888-bc20-dcb24577ca4a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000425281', 'lastCheck': '3.3', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:18,801::task::595::Storage.TaskManager.Task::(_updateState) Task=`3bb29612-ed23-4888-bc20-dcb24577ca4a`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:18,801::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:18,801::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:18,802::task::993::Storage.TaskManager.Task::(_decref) Task=`3bb29612-ed23-4888-bc20-dcb24577ca4a`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:48:18,820::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:20,442::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:20,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132825 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:22,499::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:22,533::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131287 s, 78.0 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:48:23,338::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:48:23,339::task::595::Storage.TaskManager.Task::(_updateState) Task=`6941cde1-2a6a-4e22-8e30-1853be70beba`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:23,376::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:48:23,389::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:23,390::task::1191::Storage.TaskManager.Task::(prepare) Task=`6941cde1-2a6a-4e22-8e30-1853be70beba`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:23,390::task::595::Storage.TaskManager.Task::(_updateState) Task=`6941cde1-2a6a-4e22-8e30-1853be70beba`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:23,390::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:23,390::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:23,391::task::993::Storage.TaskManager.Task::(_decref) Task=`6941cde1-2a6a-4e22-8e30-1853be70beba`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:48:23,401::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:48:23,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`5b72f671-9d91-439c-b65b-a6aa944d7ee5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:48:23,402::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:48:23,403::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7ab86c80-e0fe-4604-98c9-bffab169789a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:48:23,403::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:48:23,403::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:48:23,403::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`7ab86c80-e0fe-4604-98c9-bffab169789a`::Granted request >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:23,407::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:48:23,407::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5b72f671-9d91-439c-b65b-a6aa944d7ee5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:48:23,408::task::993::Storage.TaskManager.Task::(_decref) Task=`5b72f671-9d91-439c-b65b-a6aa944d7ee5`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:23,412::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:23,418::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:48:23,439::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952018944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:23,440::task::1191::Storage.TaskManager.Task::(prepare) Task=`5b72f671-9d91-439c-b65b-a6aa944d7ee5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852952018944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:23,440::task::595::Storage.TaskManager.Task::(_updateState) Task=`5b72f671-9d91-439c-b65b-a6aa944d7ee5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:48:23,440::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:48:23,440::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:48:23,441::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:48:23,441::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:48:23,441::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:48:23,441::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:48:23,442::task::993::Storage.TaskManager.Task::(_decref) Task=`5b72f671-9d91-439c-b65b-a6aa944d7ee5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:48:24,556::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:24,589::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131285 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:25,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:25,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:25,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:25,460::__init__::232::IOProcess::(_processLogs) (7748) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:25,460::__init__::232::IOProcess::(_processLogs) (7748) Queuing response >Thread-16::DEBUG::2014-08-11 15:48:25,461::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:48:25,483::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000440487 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:25,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:25,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:25,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:25,485::__init__::232::IOProcess::(_processLogs) (7749) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:25,485::__init__::232::IOProcess::(_processLogs) (7749) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:25,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:25,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:25,487::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:25,487::__init__::232::IOProcess::(_processLogs) (7750) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:25,488::__init__::232::IOProcess::(_processLogs) (7750) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:25,488::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:25,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:25,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:25,490::__init__::232::IOProcess::(_processLogs) (7751) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:25,490::__init__::232::IOProcess::(_processLogs) (7751) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:25,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:25,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:25,492::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:25,492::__init__::232::IOProcess::(_processLogs) (7752) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:25,493::__init__::232::IOProcess::(_processLogs) (7752) Queuing response >Thread-122::DEBUG::2014-08-11 15:48:25,885::task::595::Storage.TaskManager.Task::(_updateState) Task=`288318aa-d3e9-4d01-844f-9a3145d47277`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:48:25,885::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:48:25,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:25,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:25,887::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:25,887::__init__::232::IOProcess::(_processLogs) (7753) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:48:25,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7753) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:25,888::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:25,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:48:25,889::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:48:25,890::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:48:25,891::task::1191::Storage.TaskManager.Task::(prepare) Task=`288318aa-d3e9-4d01-844f-9a3145d47277`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:48:25,892::__init__::232::IOProcess::(_processLogs) (7754) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:48:25,892::task::595::Storage.TaskManager.Task::(_updateState) Task=`288318aa-d3e9-4d01-844f-9a3145d47277`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:48:25,893::__init__::232::IOProcess::(_processLogs) (7754) Queuing response >Thread-122::DEBUG::2014-08-11 15:48:25,894::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:48:25,894::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:48:25,894::task::993::Storage.TaskManager.Task::(_decref) Task=`288318aa-d3e9-4d01-844f-9a3145d47277`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:48:26,612::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:26,644::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133576 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:28,427::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:28,429::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:28,477::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:28,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:28,700::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133454 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:30,722::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:30,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133572 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:32,779::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:32,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013138 s, 77.9 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:48:33,454::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:48:33,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd1d436a-b1d6-4b2d-bbdc-f78504fa9a46`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:33,477::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:33,484::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:33,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:33,489::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:48:33,491::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:33,491::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd1d436a-b1d6-4b2d-bbdc-f78504fa9a46`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:33,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd1d436a-b1d6-4b2d-bbdc-f78504fa9a46`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:33,491::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:33,492::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:33,492::task::993::Storage.TaskManager.Task::(_decref) Task=`dd1d436a-b1d6-4b2d-bbdc-f78504fa9a46`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:48:33,503::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:48:33,503::task::595::Storage.TaskManager.Task::(_updateState) Task=`b94eaeae-fe3d-4f84-b691-77918fffb883`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:48:33,504::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:48:33,504::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`216f8eca-cba2-47b3-9f50-be8f073a11d1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:48:33,505::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:48:33,505::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:48:33,505::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`216f8eca-cba2-47b3-9f50-be8f073a11d1`::Granted request >Thread-34::DEBUG::2014-08-11 15:48:33,506::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b94eaeae-fe3d-4f84-b691-77918fffb883`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:48:33,506::task::993::Storage.TaskManager.Task::(_decref) Task=`b94eaeae-fe3d-4f84-b691-77918fffb883`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:48:33,526::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:33,526::task::1191::Storage.TaskManager.Task::(prepare) Task=`b94eaeae-fe3d-4f84-b691-77918fffb883`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:33,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`b94eaeae-fe3d-4f84-b691-77918fffb883`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:48:33,527::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:48:33,527::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:48:33,527::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:48:33,528::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:48:33,528::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:48:33,528::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:48:33,528::task::993::Storage.TaskManager.Task::(_decref) Task=`b94eaeae-fe3d-4f84-b691-77918fffb883`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:48:33,925::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb8ebfc0-9bbb-4fe2-ad40-ae785588802e`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:33,925::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:48:33,925::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000440487', 'lastCheck': '8.4', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:33,926::task::1191::Storage.TaskManager.Task::(prepare) Task=`bb8ebfc0-9bbb-4fe2-ad40-ae785588802e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000440487', 'lastCheck': '8.4', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:33,926::task::595::Storage.TaskManager.Task::(_updateState) Task=`bb8ebfc0-9bbb-4fe2-ad40-ae785588802e`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:33,926::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:33,926::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:33,926::task::993::Storage.TaskManager.Task::(_decref) Task=`bb8ebfc0-9bbb-4fe2-ad40-ae785588802e`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:48:33,959::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:34,957::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:34,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131872 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:35,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:35,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:35,494::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:35,494::__init__::232::IOProcess::(_processLogs) (7755) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:35,494::__init__::232::IOProcess::(_processLogs) (7755) Queuing response >Thread-16::DEBUG::2014-08-11 15:48:35,495::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:48:35,515::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000415876 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:35,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:35,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:35,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:35,517::__init__::232::IOProcess::(_processLogs) (7756) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:35,518::__init__::232::IOProcess::(_processLogs) (7756) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:35,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:35,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:35,519::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:35,520::__init__::232::IOProcess::(_processLogs) (7757) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:35,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7757) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:35,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:35,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:35,523::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:35,523::__init__::232::IOProcess::(_processLogs) (7758) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:35,523::__init__::232::IOProcess::(_processLogs) (7758) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:35,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:35,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:35,524::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:35,524::__init__::232::IOProcess::(_processLogs) (7759) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:35,525::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7759) Queuing response >Dummy-33::DEBUG::2014-08-11 15:48:37,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:37,057::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0225593 s, 45.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:38,496::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:38,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:38,577::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:39,092::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:39,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131502 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:41,148::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:41,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132749 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:43,204::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:43,238::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132514 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:48:43,541::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:48:43,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`46d186ad-6147-4dbf-8348-59a5a0fbe965`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:43,576::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:43,584::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:43,586::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:43,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:48:43,590::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:43,590::task::1191::Storage.TaskManager.Task::(prepare) Task=`46d186ad-6147-4dbf-8348-59a5a0fbe965`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:43,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`46d186ad-6147-4dbf-8348-59a5a0fbe965`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:43,591::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:43,591::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:43,591::task::993::Storage.TaskManager.Task::(_decref) Task=`46d186ad-6147-4dbf-8348-59a5a0fbe965`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:48:43,601::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:48:43,602::task::595::Storage.TaskManager.Task::(_updateState) Task=`416f55fd-fa2e-41db-aa97-46ae7450ef5f`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:48:43,602::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:48:43,603::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6e0df8a4-b7aa-46db-a288-b591cfd1a3d5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:48:43,603::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:48:43,604::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:48:43,604::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6e0df8a4-b7aa-46db-a288-b591cfd1a3d5`::Granted request >Thread-34::DEBUG::2014-08-11 15:48:43,604::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`416f55fd-fa2e-41db-aa97-46ae7450ef5f`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:48:43,605::task::993::Storage.TaskManager.Task::(_decref) Task=`416f55fd-fa2e-41db-aa97-46ae7450ef5f`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:48:43,624::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:43,625::task::1191::Storage.TaskManager.Task::(prepare) Task=`416f55fd-fa2e-41db-aa97-46ae7450ef5f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:43,625::task::595::Storage.TaskManager.Task::(_updateState) Task=`416f55fd-fa2e-41db-aa97-46ae7450ef5f`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:48:43,625::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:48:43,625::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:48:43,626::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:48:43,626::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:48:43,626::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:48:43,627::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:48:43,627::task::993::Storage.TaskManager.Task::(_decref) Task=`416f55fd-fa2e-41db-aa97-46ae7450ef5f`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:48:45,260::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:45,294::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132517 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:45,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:45,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:45,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:48:45,530::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:48:45,530::__init__::232::IOProcess::(_processLogs) (7760) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:45,549::__init__::232::IOProcess::(_processLogs) (7760) Queuing response >Thread-16::DEBUG::2014-08-11 15:48:45,551::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000498161 s, 913 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:45,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:45,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:45,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:45,553::__init__::232::IOProcess::(_processLogs) (7761) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:45,554::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7761) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:45,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:45,556::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:45,556::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:45,556::__init__::232::IOProcess::(_processLogs) (7762) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:45,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7762) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:45,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:45,558::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:45,558::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:45,558::__init__::232::IOProcess::(_processLogs) (7763) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:45,559::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7763) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:45,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:45,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:45,560::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:45,561::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(7764) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:45,562::__init__::232::IOProcess::(_processLogs) (7764) Queuing response >Dummy-33::DEBUG::2014-08-11 15:48:47,317::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:47,351::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131295 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:48,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:48,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:48,677::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:48:49,053::task::595::Storage.TaskManager.Task::(_updateState) Task=`543bc27d-604d-4bfa-a029-8f9b07a61a3c`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:49,053::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:48:49,054::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000498161', 'lastCheck': '3.5', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:49,054::task::1191::Storage.TaskManager.Task::(prepare) Task=`543bc27d-604d-4bfa-a029-8f9b07a61a3c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000498161', 'lastCheck': '3.5', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:48:49,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`543bc27d-604d-4bfa-a029-8f9b07a61a3c`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:49,054::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:49,055::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:49,055::task::993::Storage.TaskManager.Task::(_decref) Task=`543bc27d-604d-4bfa-a029-8f9b07a61a3c`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:48:49,078::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:49,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:49,407::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133818 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:51,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:51,463::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134421 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:48:53,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:53,520::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132454 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:48:53,639::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:48:53,640::task::595::Storage.TaskManager.Task::(_updateState) Task=`a975583f-5fcc-4e35-b5b1-90bd49990de7`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:48:53,676::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:53,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:53,686::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:53,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:48:53,689::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:53,690::task::1191::Storage.TaskManager.Task::(prepare) Task=`a975583f-5fcc-4e35-b5b1-90bd49990de7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:48:53,690::task::595::Storage.TaskManager.Task::(_updateState) Task=`a975583f-5fcc-4e35-b5b1-90bd49990de7`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:48:53,690::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:48:53,690::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:48:53,690::task::993::Storage.TaskManager.Task::(_decref) Task=`a975583f-5fcc-4e35-b5b1-90bd49990de7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:48:53,701::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:48:53,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7d9b218-d16f-486b-a166-54d0f795e7d4`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:48:53,702::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:48:53,703::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`15c375bb-e8be-427c-8799-9dc6ac006f2c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:48:53,703::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:48:53,703::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:48:53,704::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`15c375bb-e8be-427c-8799-9dc6ac006f2c`::Granted request >Thread-34::DEBUG::2014-08-11 15:48:53,704::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e7d9b218-d16f-486b-a166-54d0f795e7d4`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:48:53,704::task::993::Storage.TaskManager.Task::(_decref) Task=`e7d9b218-d16f-486b-a166-54d0f795e7d4`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:48:53,725::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:53,725::task::1191::Storage.TaskManager.Task::(prepare) Task=`e7d9b218-d16f-486b-a166-54d0f795e7d4`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:48:53,725::task::595::Storage.TaskManager.Task::(_updateState) Task=`e7d9b218-d16f-486b-a166-54d0f795e7d4`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:48:53,725::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:48:53,726::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:48:53,726::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:48:53,726::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:48:53,727::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:48:53,727::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:48:53,727::task::993::Storage.TaskManager.Task::(_decref) Task=`e7d9b218-d16f-486b-a166-54d0f795e7d4`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:48:55,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:48:55,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:55,565::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:55,565::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:55,566::__init__::232::IOProcess::(_processLogs) (7765) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:55,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(7765) Queuing response >Thread-16::DEBUG::2014-08-11 15:48:55,574::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:48:55,597::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000406869 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:55,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:55,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Dummy-33::DEBUG::2014-08-11 15:48:55,603::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.039734 s, 25.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:48:55,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:55,606::__init__::232::IOProcess::(_processLogs) (7766) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:48:55,608::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(7766) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:55,611::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:55,612::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:55,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:55,614::__init__::232::IOProcess::(_processLogs) (7767) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:55,617::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(7767) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:55,618::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:55,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:55,620::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:55,621::__init__::232::IOProcess::(_processLogs) (7768) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:55,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(7768) Queuing response >Thread-14::DEBUG::2014-08-11 15:48:55,625::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:48:55,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:48:55,628::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:48:55,629::__init__::232::IOProcess::(_processLogs) (7769) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:48:55,631::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|(7769) Queuing response >Dummy-33::DEBUG::2014-08-11 15:48:57,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:57,685::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131019 s, 78.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:58,692::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:58,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:48:58,777::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:48:59,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:48:59,740::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133763 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:01,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:01,797::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134443 s, 76.2 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:49:03,739::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:49:03,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`123b7876-a08a-4093-b51a-ada429b31585`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:03,776::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:03,783::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:03,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:03,786::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:49:03,789::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:03,790::task::1191::Storage.TaskManager.Task::(prepare) Task=`123b7876-a08a-4093-b51a-ada429b31585`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:03,790::task::595::Storage.TaskManager.Task::(_updateState) Task=`123b7876-a08a-4093-b51a-ada429b31585`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:03,790::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:03,790::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:03,790::task::993::Storage.TaskManager.Task::(_decref) Task=`123b7876-a08a-4093-b51a-ada429b31585`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:49:03,801::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:49:03,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f7442b5-efd4-441e-8fb4-5fde8c4da8cd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:49:03,802::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:49:03,803::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a4d62eb6-a4a8-44e0-8216-aae4ae2f536b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:49:03,803::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:49:03,804::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:49:03,804::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a4d62eb6-a4a8-44e0-8216-aae4ae2f536b`::Granted request >Thread-34::DEBUG::2014-08-11 15:49:03,804::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9f7442b5-efd4-441e-8fb4-5fde8c4da8cd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:49:03,805::task::993::Storage.TaskManager.Task::(_decref) Task=`9f7442b5-efd4-441e-8fb4-5fde8c4da8cd`::ref 1 aborting False >Dummy-33::DEBUG::2014-08-11 15:49:03,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::INFO::2014-08-11 15:49:03,840::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:03,841::task::1191::Storage.TaskManager.Task::(prepare) Task=`9f7442b5-efd4-441e-8fb4-5fde8c4da8cd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951953408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:03,841::task::595::Storage.TaskManager.Task::(_updateState) Task=`9f7442b5-efd4-441e-8fb4-5fde8c4da8cd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:49:03,842::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:49:03,842::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:49:03,843::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:49:03,844::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:49:03,845::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:49:03,845::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:49:03,846::task::993::Storage.TaskManager.Task::(_decref) Task=`9f7442b5-efd4-441e-8fb4-5fde8c4da8cd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:49:03,855::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132841 s, 77.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:49:04,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`7bdb6b9d-ed17-4198-a6ce-d1e9119d4ca6`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:04,168::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:49:04,169::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406869', 'lastCheck': '8.5', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:04,169::task::1191::Storage.TaskManager.Task::(prepare) Task=`7bdb6b9d-ed17-4198-a6ce-d1e9119d4ca6`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000406869', 'lastCheck': '8.5', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:04,169::task::595::Storage.TaskManager.Task::(_updateState) Task=`7bdb6b9d-ed17-4198-a6ce-d1e9119d4ca6`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:04,169::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:04,170::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:04,170::task::993::Storage.TaskManager.Task::(_decref) Task=`7bdb6b9d-ed17-4198-a6ce-d1e9119d4ca6`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:49:04,189::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:49:05,633::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:05,633::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:05,633::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:05,634::__init__::232::IOProcess::(_processLogs) (7770) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:05,634::__init__::232::IOProcess::(_processLogs) (7770) Queuing response >Thread-16::DEBUG::2014-08-11 15:49:05,635::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:49:05,654::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419817 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:05,655::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:05,656::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUGDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:05,656::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:05,656::__init__::232::IOProcess::(_processLogs) (7771) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:05,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:05,658::__init__::232::IOProcess::(_processLogs) (7772) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:05,659::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:05,659::__init__::232::IOProcess::(_processLogs) (7773) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:05,660::__init__::232::IOProcess::(_processLogs) (7773) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:05,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:05,661::__init__::232::IOProcess::(_processLogs) (7774) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:05,661::__init__::232::IOProcess::(_processLogs) (7774) Queuing response >Dummy-33::DEBUG::2014-08-11 15:49:05,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:05,911::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131351 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:07,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:07,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132667 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:08,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:08,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:08,878::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:09,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:10,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132066 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:12,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:12,080::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013164 s, 77.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:49:13,858::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:49:13,859::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7d6d5ea-ec66-4449-9679-445bf70c2b33`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:13,876::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:13,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:13,887::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:13,888::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:49:13,889::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:13,889::task::1191::Storage.TaskManager.Task::(prepare) Task=`f7d6d5ea-ec66-4449-9679-445bf70c2b33`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:13,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`f7d6d5ea-ec66-4449-9679-445bf70c2b33`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:13,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:13,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:13,890::task::993::Storage.TaskManager.Task::(_decref) Task=`f7d6d5ea-ec66-4449-9679-445bf70c2b33`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:49:13,900::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:49:13,901::task::595::Storage.TaskManager.Task::(_updateState) Task=`01d15b7d-c6d0-4263-9207-5721551a6839`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:49:13,901::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:49:13,902::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d4ddaf1-bbd5-49a5-86dd-61d86bb8ece1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:49:13,902::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:49:13,903::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:49:13,903::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d4ddaf1-bbd5-49a5-86dd-61d86bb8ece1`::Granted request >Thread-34::DEBUG::2014-08-11 15:49:13,903::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`01d15b7d-c6d0-4263-9207-5721551a6839`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:49:13,904::task::993::Storage.TaskManager.Task::(_decref) Task=`01d15b7d-c6d0-4263-9207-5721551a6839`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:49:13,923::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:13,924::task::1191::Storage.TaskManager.Task::(prepare) Task=`01d15b7d-c6d0-4263-9207-5721551a6839`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:13,924::task::595::Storage.TaskManager.Task::(_updateState) Task=`01d15b7d-c6d0-4263-9207-5721551a6839`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:49:13,924::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:49:13,924::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:49:13,925::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:49:13,925::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:49:13,925::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:49:13,925::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:49:13,926::task::993::Storage.TaskManager.Task::(_decref) Task=`01d15b7d-c6d0-4263-9207-5721551a6839`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:49:14,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:14,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013225 s, 77.4 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:15,663::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:15,664::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:49:15,665::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:49:15,666::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:15,685::__init__::232::IOProcess::(_processLogs) (7775) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:15,686::__init__::232::IOProcess::(_processLogs) (7775) Queuing response >Thread-16::DEBUG::2014-08-11 15:49:15,687::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000534329 s, 852 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:15,688::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:15,688::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:15,688::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:15,689::__init__::232::IOProcess::(_processLogs) (7776) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:15,689::__init__::232::IOProcess::(_processLogs) (7776) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:15,690::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:15,691::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:15,691::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:15,692::__init__::232::IOProcess::(_processLogs) (7777) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:15,692::__init__::232::IOProcess::(_processLogs) (7777) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:15,692::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:15,693::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:15,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:15,694::__init__::232::IOProcess::(_processLogs) (7778) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:15,694::__init__::232::IOProcess::(_processLogs) (7778) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:15,694::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:15,695::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:15,696::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:15,696::__init__::232::IOProcess::(_processLogs) (7779) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:15,697::__init__::232::IOProcess::(_processLogs) (7779) Queuing response >Dummy-33::DEBUG::2014-08-11 15:49:16,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:16,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132167 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:18,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:18,249::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132399 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:18,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:18,896::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:18,977::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:49:19,280::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0dd3c8c-6546-4849-9783-8f9ca2402f0b`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:19,280::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:49:19,280::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000534329', 'lastCheck': '3.6', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:19,281::task::1191::Storage.TaskManager.Task::(prepare) Task=`f0dd3c8c-6546-4849-9783-8f9ca2402f0b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000534329', 'lastCheck': '3.6', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:19,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0dd3c8c-6546-4849-9783-8f9ca2402f0b`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:19,281::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:19,281::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:19,282::task::993::Storage.TaskManager.Task::(_decref) Task=`f0dd3c8c-6546-4849-9783-8f9ca2402f0b`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:49:19,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:20,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:20,320::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0223764 s, 45.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:22,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:22,486::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131076 s, 78.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:49:23,940::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:49:23,941::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c2238b9-8bef-4b1f-9473-8e7ff084ea84`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:23,976::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:23,983::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:23,986::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:23,987::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:49:23,989::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:23,990::task::1191::Storage.TaskManager.Task::(prepare) Task=`7c2238b9-8bef-4b1f-9473-8e7ff084ea84`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:23,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`7c2238b9-8bef-4b1f-9473-8e7ff084ea84`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:23,990::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:23,990::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:23,990::task::993::Storage.TaskManager.Task::(_decref) Task=`7c2238b9-8bef-4b1f-9473-8e7ff084ea84`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:49:24,001::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:49:24,002::task::595::Storage.TaskManager.Task::(_updateState) Task=`a564d8aa-0f46-47b5-9f96-53f2b20b6e6e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:49:24,002::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:49:24,003::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4bef2bb1-1cb6-40cf-93c4-12e18743c490`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:49:24,003::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:49:24,003::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:49:24,004::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4bef2bb1-1cb6-40cf-93c4-12e18743c490`::Granted request >Thread-34::DEBUG::2014-08-11 15:49:24,004::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a564d8aa-0f46-47b5-9f96-53f2b20b6e6e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:49:24,004::task::993::Storage.TaskManager.Task::(_decref) Task=`a564d8aa-0f46-47b5-9f96-53f2b20b6e6e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:49:24,025::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:24,025::task::1191::Storage.TaskManager.Task::(prepare) Task=`a564d8aa-0f46-47b5-9f96-53f2b20b6e6e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:24,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`a564d8aa-0f46-47b5-9f96-53f2b20b6e6e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:49:24,026::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:49:24,026::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:49:24,026::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:49:24,027::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:49:24,027::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:49:24,027::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:49:24,027::task::993::Storage.TaskManager.Task::(_decref) Task=`a564d8aa-0f46-47b5-9f96-53f2b20b6e6e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:49:24,511::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:24,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133475 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:25,697::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:25,698::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:49:25,699::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:49:25,699::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:25,719::__init__::232::IOProcess::(_processLogs) (7780) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:25,719::__init__::232::IOProcess::(_processLogs) (7780) Queuing response >Thread-16::DEBUG::2014-08-11 15:49:25,720::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000621339 s, 732 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:25,721::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:25,721::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:25,722::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:25,722::__init__::232::IOProcess::(_processLogs) (7781) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:25,723::__init__::232::IOProcess::(_processLogs) DEBUG|(7781) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:25,724::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:25,724::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:25,725::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:25,725::__init__::232::IOProcess::(_processLogs) (7782) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:25,725::__init__::232::IOProcess::(_processLogs) DEBUG|(7782) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:25,726::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:25,726::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:25,727::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:25,727::__init__::232::IOProcess::(_processLogs) (7783) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:25,727::__init__::232::IOProcess::(_processLogs) (7783) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:25,728::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:25,728::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:25,729::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:25,729::__init__::232::IOProcess::(_processLogs) (7784) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:25,729::__init__::232::IOProcess::(_processLogs) DEBUG|(7784) Queuing response >Thread-122::DEBUG::2014-08-11 15:49:26,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcb07afd-31e4-4a58-ac64-07907618b295`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:49:26,026::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:49:26,027::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:26,028::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:26,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:26,028::__init__::232::IOProcess::(_processLogs) (7785) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:49:26,029::__init__::232::IOProcess::(_processLogs) DEBUG|(7785) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:26,030::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:26,030::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:49:26,031::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:49:26,032::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:49:26,032::task::1191::Storage.TaskManager.Task::(prepare) Task=`dcb07afd-31e4-4a58-ac64-07907618b295`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:49:26,033::__init__::232::IOProcess::(_processLogs) (7786) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:49:26,033::task::595::Storage.TaskManager.Task::(_updateState) Task=`dcb07afd-31e4-4a58-ac64-07907618b295`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:49:26,034::__init__::232::IOProcess::(_processLogs) (7786) Queuing response >Thread-122::DEBUG::2014-08-11 15:49:26,035::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:49:26,035::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:49:26,036::task::993::Storage.TaskManager.Task::(_decref) Task=`dcb07afd-31e4-4a58-ac64-07907618b295`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:49:26,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:26,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133942 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:28,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:28,659::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133591 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:28,994::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:28,995::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:29,077::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:30,682::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:30,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132815 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:32,739::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:32,790::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276182 s, 37.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:49:34,039::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:49:34,040::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7cda56c-016e-4883-9415-3606faaaf5e0`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:34,077::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:34,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:34,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:34,090::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:49:34,091::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:34,092::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7cda56c-016e-4883-9415-3606faaaf5e0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:34,092::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7cda56c-016e-4883-9415-3606faaaf5e0`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:34,092::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:34,093::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:34,093::task::993::Storage.TaskManager.Task::(_decref) Task=`d7cda56c-016e-4883-9415-3606faaaf5e0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:49:34,103::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:49:34,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e546a73-4ea7-4ef8-b7db-c5cdf72c68ac`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:49:34,104::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:49:34,104::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a7e3ae2-b797-4072-a6be-2da8bac445ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:49:34,105::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:49:34,105::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:49:34,105::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a7e3ae2-b797-4072-a6be-2da8bac445ea`::Granted request >Thread-34::DEBUG::2014-08-11 15:49:34,106::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8e546a73-4ea7-4ef8-b7db-c5cdf72c68ac`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:49:34,106::task::993::Storage.TaskManager.Task::(_decref) Task=`8e546a73-4ea7-4ef8-b7db-c5cdf72c68ac`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:49:34,126::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:34,126::task::1191::Storage.TaskManager.Task::(prepare) Task=`8e546a73-4ea7-4ef8-b7db-c5cdf72c68ac`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:34,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`8e546a73-4ea7-4ef8-b7db-c5cdf72c68ac`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:49:34,127::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:49:34,127::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:49:34,127::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:49:34,128::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:49:34,128::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:49:34,128::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:49:34,128::task::993::Storage.TaskManager.Task::(_decref) Task=`8e546a73-4ea7-4ef8-b7db-c5cdf72c68ac`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:49:34,388::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3468e80-9244-4823-ba94-13a4ede4469a`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:34,389::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:49:34,389::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000621339', 'lastCheck': '8.7', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:34,389::task::1191::Storage.TaskManager.Task::(prepare) Task=`a3468e80-9244-4823-ba94-13a4ede4469a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000621339', 'lastCheck': '8.7', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:34,390::task::595::Storage.TaskManager.Task::(_updateState) Task=`a3468e80-9244-4823-ba94-13a4ede4469a`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:34,390::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:34,390::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:34,390::task::993::Storage.TaskManager.Task::(_decref) Task=`a3468e80-9244-4823-ba94-13a4ede4469a`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:49:34,411::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:34,953::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:34,987::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132581 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:35,731::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:35,732::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:49:35,733::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:49:35,734::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:35,753::__init__::232::IOProcess::(_processLogs) (7787) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:35,753::__init__::232::IOProcess::(_processLogs) (7787) Queuing response >Thread-16::DEBUG::2014-08-11 15:49:35,754::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00079552 s, 572 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:35,755::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:35,755::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:35,756::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:35,756::__init__::232::IOProcess::(_processLogs) (7788) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:35,757::__init__::232::IOProcess::(_processLogs) DEBUG|(7788) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:35,758::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:35,758::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:35,759::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:35,759::__init__::232::IOProcess::(_processLogs) (7789) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:35,759::__init__::232::IOProcess::(_processLogs) DEBUG|(7789) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:35,760::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:35,761::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:35,761::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:35,762::__init__::232::IOProcess::(_processLogs) (7790) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:35,762::__init__::232::IOProcess::(_processLogs) (7790) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:35,762::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:35,763::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:35,763::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:35,763::__init__::232::IOProcess::(_processLogs) (7791) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:35,764::__init__::232::IOProcess::(_processLogs) DEBUG|(7791) Queuing response >Dummy-33::DEBUG::2014-08-11 15:49:37,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:37,043::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132013 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:39,065::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:39,098::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:39,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:39,101::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:39,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189762 s, 54.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:41,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:41,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133436 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:43,184::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:43,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132939 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:44,108::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:44,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:49:44,141::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:49:44,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`676106ed-79d6-4592-88bb-eb38253d627f`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:44,182::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:44,182::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:49:44,196::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:44,197::task::1191::Storage.TaskManager.Task::(prepare) Task=`676106ed-79d6-4592-88bb-eb38253d627f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:44,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`676106ed-79d6-4592-88bb-eb38253d627f`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:44,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:44,198::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:44,198::task::993::Storage.TaskManager.Task::(_decref) Task=`676106ed-79d6-4592-88bb-eb38253d627f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:49:44,208::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:49:44,208::task::595::Storage.TaskManager.Task::(_updateState) Task=`15b8553c-6707-461c-8df5-0cacb6909321`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:49:44,209::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:49:44,209::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`acf99a01-1912-4b85-8874-c8a6d2427ed6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:49:44,210::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:49:44,210::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:49:44,210::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`acf99a01-1912-4b85-8874-c8a6d2427ed6`::Granted request >Thread-34::DEBUG::2014-08-11 15:49:44,211::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`15b8553c-6707-461c-8df5-0cacb6909321`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:49:44,211::task::993::Storage.TaskManager.Task::(_decref) Task=`15b8553c-6707-461c-8df5-0cacb6909321`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:49:44,231::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:44,231::task::1191::Storage.TaskManager.Task::(prepare) Task=`15b8553c-6707-461c-8df5-0cacb6909321`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:44,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`15b8553c-6707-461c-8df5-0cacb6909321`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:49:44,231::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:49:44,232::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:49:44,232::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:49:44,232::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:49:44,233::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:49:44,233::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:49:44,233::task::993::Storage.TaskManager.Task::(_decref) Task=`15b8553c-6707-461c-8df5-0cacb6909321`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:49:45,256::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:45,290::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131807 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:45,766::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:45,767::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:49:45,768::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:49:45,769::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:45,788::__init__::232::IOProcess::(_processLogs) (7792) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:45,788::__init__::232::IOProcess::(_processLogs) (7792) Queuing response >Thread-16::DEBUG::2014-08-11 15:49:45,790::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419063 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:45,791::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:45,791::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:45,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:45,792::__init__::232::IOProcess::(_processLogs) (7793) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:45,793::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:45,793::__init__::232::IOProcess::(_processLogs) (7794) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:45,794::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:45,795::__init__::232::IOProcess::(_processLogs) (7795) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:45,795::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:45,796::__init__::232::IOProcess::(_processLogs) (7796) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:45,796::__init__::232::IOProcess::(_processLogs) (7796) Queuing response >Dummy-33::DEBUG::2014-08-11 15:49:47,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:47,346::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131646 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:49,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:49,187::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:49,278::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:49,368::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:49,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133365 s, 76.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:49:49,502::task::595::Storage.TaskManager.Task::(_updateState) Task=`7619fa67-85bd-40ec-a50c-0f410730f3a8`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:49,502::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:49:49,503::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419063', 'lastCheck': '3.7', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:49,503::task::1191::Storage.TaskManager.Task::(prepare) Task=`7619fa67-85bd-40ec-a50c-0f410730f3a8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419063', 'lastCheck': '3.7', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:49:49,503::task::595::Storage.TaskManager.Task::(_updateState) Task=`7619fa67-85bd-40ec-a50c-0f410730f3a8`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:49,503::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:49,504::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:49,504::task::993::Storage.TaskManager.Task::(_decref) Task=`7619fa67-85bd-40ec-a50c-0f410730f3a8`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:49:49,523::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:51,425::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:51,459::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133018 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:49:53,483::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:53,516::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130904 s, 78.2 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:49:54,245::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:49:54,246::task::595::Storage.TaskManager.Task::(_updateState) Task=`85a13a4f-5817-4c62-8a93-e7b4a85e7f16`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:49:54,277::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:54,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:54,287::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:54,288::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:49:54,290::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:54,290::task::1191::Storage.TaskManager.Task::(prepare) Task=`85a13a4f-5817-4c62-8a93-e7b4a85e7f16`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:49:54,291::task::595::Storage.TaskManager.Task::(_updateState) Task=`85a13a4f-5817-4c62-8a93-e7b4a85e7f16`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:49:54,291::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:49:54,291::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:49:54,291::task::993::Storage.TaskManager.Task::(_decref) Task=`85a13a4f-5817-4c62-8a93-e7b4a85e7f16`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:49:54,302::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:49:54,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdfe385d-8285-4754-a786-b3e4ccc3fedc`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:49:54,303::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:49:54,303::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2f86d0fe-bb03-4310-99df-e3ef1c67d952`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:49:54,304::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:49:54,304::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:49:54,304::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2f86d0fe-bb03-4310-99df-e3ef1c67d952`::Granted request >Thread-34::DEBUG::2014-08-11 15:49:54,307::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bdfe385d-8285-4754-a786-b3e4ccc3fedc`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:49:54,307::task::993::Storage.TaskManager.Task::(_decref) Task=`bdfe385d-8285-4754-a786-b3e4ccc3fedc`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:49:54,327::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:54,327::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdfe385d-8285-4754-a786-b3e4ccc3fedc`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:49:54,327::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdfe385d-8285-4754-a786-b3e4ccc3fedc`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:49:54,328::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:49:54,328::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:49:54,328::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:49:54,329::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:49:54,329::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:49:54,329::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:49:54,329::task::993::Storage.TaskManager.Task::(_decref) Task=`bdfe385d-8285-4754-a786-b3e4ccc3fedc`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:49:55,539::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:55,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135911 s, 75.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:55,798::__init__::232::IOProcess::(_processLogs) (7796) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:55,799::__init__::232::IOProcess::(_processLogs) (7796) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:49:55,800::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:49:55,801::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:55,820::__init__::232::IOProcess::(_processLogs) (7797) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:55,821::__init__::232::IOProcess::(_processLogs) (7797) Queuing response >Thread-16::DEBUG::2014-08-11 15:49:55,822::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000646066 s, 704 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:49:55,823::__init__::232::IOProcess::(_processLogs) (7796) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:55,823::__init__::232::IOProcess::(_processLogs) (7796) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:55,823::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:55,824::__init__::232::IOProcess::(_processLogs) (7798) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:49:55,825::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:55,825::__init__::232::IOProcess::(_processLogs) (7799) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:55,825::__init__::232::IOProcess::(_processLogs) (7799) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:55,826::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:55,827::__init__::232::IOProcess::(_processLogs) (7800) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:55,827::__init__::232::IOProcess::(_processLogs) DEBUG|(7800) Queuing response >Thread-14::DEBUG::2014-08-11 15:49:55,828::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:49:55,828::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:49:55,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:49:55,830::__init__::232::IOProcess::(_processLogs) (7801) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:49:55,830::__init__::232::IOProcess::(_processLogs) (7801) Queuing response >Dummy-33::DEBUG::2014-08-11 15:49:57,600::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:57,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131965 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:59,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:59,297::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:49:59,378::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:49:59,657::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:49:59,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132773 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:01,714::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:01,748::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132793 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:03,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:03,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132489 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:50:04,342::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:50:04,343::task::595::Storage.TaskManager.Task::(_updateState) Task=`381bccf1-8d4a-47a8-9958-81574f5c2b57`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:04,377::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:04,386::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:04,387::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:04,388::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:50:04,391::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:04,391::task::1191::Storage.TaskManager.Task::(prepare) Task=`381bccf1-8d4a-47a8-9958-81574f5c2b57`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:04,391::task::595::Storage.TaskManager.Task::(_updateState) Task=`381bccf1-8d4a-47a8-9958-81574f5c2b57`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:04,391::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:04,392::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:04,392::task::993::Storage.TaskManager.Task::(_decref) Task=`381bccf1-8d4a-47a8-9958-81574f5c2b57`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:50:04,403::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:50:04,404::task::595::Storage.TaskManager.Task::(_updateState) Task=`342f46b4-3e47-4734-b3f9-5cb6247b3a52`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:50:04,404::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:50:04,405::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`907f87c0-d8ab-42e8-b6c8-cda3a832d02d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:50:04,405::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:50:04,406::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:50:04,406::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`907f87c0-d8ab-42e8-b6c8-cda3a832d02d`::Granted request >Thread-34::DEBUG::2014-08-11 15:50:04,406::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`342f46b4-3e47-4734-b3f9-5cb6247b3a52`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:50:04,407::task::993::Storage.TaskManager.Task::(_decref) Task=`342f46b4-3e47-4734-b3f9-5cb6247b3a52`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:50:04,426::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:04,427::task::1191::Storage.TaskManager.Task::(prepare) Task=`342f46b4-3e47-4734-b3f9-5cb6247b3a52`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:04,427::task::595::Storage.TaskManager.Task::(_updateState) Task=`342f46b4-3e47-4734-b3f9-5cb6247b3a52`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:50:04,427::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:50:04,428::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:50:04,428::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:50:04,428::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:50:04,428::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:50:04,429::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:50:04,429::task::993::Storage.TaskManager.Task::(_decref) Task=`342f46b4-3e47-4734-b3f9-5cb6247b3a52`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:50:04,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`47a3da43-54b4-444d-9fb8-41541d0a28b9`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:04,612::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:50:04,613::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000646066', 'lastCheck': '8.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:04,613::task::1191::Storage.TaskManager.Task::(prepare) Task=`47a3da43-54b4-444d-9fb8-41541d0a28b9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000646066', 'lastCheck': '8.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:04,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`47a3da43-54b4-444d-9fb8-41541d0a28b9`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:04,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:04,614::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:04,614::task::993::Storage.TaskManager.Task::(_decref) Task=`47a3da43-54b4-444d-9fb8-41541d0a28b9`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:50:04,638::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:50:05,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-14::DEBUG::2014-08-11 15:50:05,848::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:05,849::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:05,851::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:50:05,851::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:50:05,852::__init__::232::IOProcess::(_processLogs) (7802) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:05,853::__init__::232::IOProcess::(_processLogs) (7802) Queuing response >Dummy-33::DEBUG::2014-08-11 15:50:05,881::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0323607 s, 31.6 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:50:05,883::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00045044 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:05,886::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:05,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:05,889::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:05,890::__init__::232::IOProcess::(_processLogs) (7803) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:05,892::__init__::232::IOProcess::(_processLogs) DEBUG|(7803) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:05,895::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:05,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:05,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:05,900::__init__::232::IOProcess::(_processLogs) (7804) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:05,901::__init__::232::IOProcess::(_processLogs) (7804) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:05,904::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:05,905::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:05,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:05,908::__init__::232::IOProcess::(_processLogs) (7805) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:05,909::__init__::232::IOProcess::(_processLogs) (7805) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:05,912::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:05,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:05,915::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:05,916::__init__::232::IOProcess::(_processLogs) (7806) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:05,918::__init__::232::IOProcess::(_processLogs) (7806) Queuing response >Dummy-33::DEBUG::2014-08-11 15:50:07,937::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:07,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134713 s, 76.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:09,395::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:09,397::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:09,478::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:50:09,993::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:10,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013213 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:12,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:12,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131914 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:14,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:14,140::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132409 s, 77.3 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:50:14,443::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:50:14,444::task::595::Storage.TaskManager.Task::(_updateState) Task=`921faa8c-51e4-4737-9059-4b7166a30135`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:14,477::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:14,486::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:14,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:14,488::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:50:14,490::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:14,491::task::1191::Storage.TaskManager.Task::(prepare) Task=`921faa8c-51e4-4737-9059-4b7166a30135`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:14,491::task::595::Storage.TaskManager.Task::(_updateState) Task=`921faa8c-51e4-4737-9059-4b7166a30135`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:14,491::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:14,491::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:14,492::task::993::Storage.TaskManager.Task::(_decref) Task=`921faa8c-51e4-4737-9059-4b7166a30135`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:50:14,502::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:50:14,503::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad7c9c72-14b7-4214-b0df-d912ac7d2439`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:50:14,503::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:50:14,504::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6d879821-39e2-447d-b61f-2072e755f1f8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:50:14,504::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:50:14,504::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:50:14,505::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6d879821-39e2-447d-b61f-2072e755f1f8`::Granted request >Thread-34::DEBUG::2014-08-11 15:50:14,505::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ad7c9c72-14b7-4214-b0df-d912ac7d2439`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:50:14,506::task::993::Storage.TaskManager.Task::(_decref) Task=`ad7c9c72-14b7-4214-b0df-d912ac7d2439`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:50:14,525::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:14,526::task::1191::Storage.TaskManager.Task::(prepare) Task=`ad7c9c72-14b7-4214-b0df-d912ac7d2439`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:14,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`ad7c9c72-14b7-4214-b0df-d912ac7d2439`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:50:14,526::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:50:14,526::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:50:14,527::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:50:14,527::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:50:14,527::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:50:14,528::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:50:14,528::task::993::Storage.TaskManager.Task::(_decref) Task=`ad7c9c72-14b7-4214-b0df-d912ac7d2439`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:50:15,914::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:15,915::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:15,917::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:50:15,917::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:50:15,918::__init__::232::IOProcess::(_processLogs) (7807) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:15,936::__init__::232::IOProcess::(_processLogs) (7807) Queuing response >Thread-16::DEBUG::2014-08-11 15:50:15,937::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000445434 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:15,938::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:15,938::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:15,939::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:15,939::__init__::232::IOProcess::(_processLogs) (7808) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:15,940::__init__::232::IOProcess::(_processLogs) DEBUG|(7808) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:15,941::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:15,941::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:15,942::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:15,942::__init__::232::IOProcess::(_processLogs) (7809) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:15,942::__init__::232::IOProcess::(_processLogs) DEBUG|(7809) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:15,943::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:15,944::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:15,944::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:15,944::__init__::232::IOProcess::(_processLogs) (7810) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:15,945::__init__::232::IOProcess::(_processLogs) DEBUG|(7810) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:15,946::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:15,946::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:15,946::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:15,947::__init__::232::IOProcess::(_processLogs) (7811) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:15,947::__init__::232::IOProcess::(_processLogs) DEBUG|(7811) Queuing response >Dummy-33::DEBUG::2014-08-11 15:50:16,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:16,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132029 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:18,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:18,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133329 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:19,495::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:19,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:19,578::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:50:19,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`2256aef4-7705-4972-a44a-e6b79f937399`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:19,734::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:50:19,735::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000445434', 'lastCheck': '3.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:19,735::task::1191::Storage.TaskManager.Task::(prepare) Task=`2256aef4-7705-4972-a44a-e6b79f937399`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000445434', 'lastCheck': '3.8', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:19,735::task::595::Storage.TaskManager.Task::(_updateState) Task=`2256aef4-7705-4972-a44a-e6b79f937399`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:19,735::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:19,736::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:19,736::task::993::Storage.TaskManager.Task::(_decref) Task=`2256aef4-7705-4972-a44a-e6b79f937399`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:50:19,756::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:50:20,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:20,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132325 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:22,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:22,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013168 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:24,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:24,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131752 s, 77.7 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:50:24,541::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:50:24,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`80bbdbf3-5ee7-4f0d-b249-3796f23ca16b`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:24,577::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:24,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:24,586::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:24,587::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:50:24,590::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:24,591::task::1191::Storage.TaskManager.Task::(prepare) Task=`80bbdbf3-5ee7-4f0d-b249-3796f23ca16b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:24,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`80bbdbf3-5ee7-4f0d-b249-3796f23ca16b`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:24,591::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:24,591::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:24,591::task::993::Storage.TaskManager.Task::(_decref) Task=`80bbdbf3-5ee7-4f0d-b249-3796f23ca16b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:50:24,602::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:50:24,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`75016a21-7418-41ea-82e8-5a4329aba567`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:50:24,603::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:50:24,604::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`14caaca0-8f86-4064-905c-1857eac61c96`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:50:24,604::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:50:24,605::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:50:24,605::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`14caaca0-8f86-4064-905c-1857eac61c96`::Granted request >Thread-34::DEBUG::2014-08-11 15:50:24,606::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`75016a21-7418-41ea-82e8-5a4329aba567`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:50:24,606::task::993::Storage.TaskManager.Task::(_decref) Task=`75016a21-7418-41ea-82e8-5a4329aba567`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:50:24,625::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:24,626::task::1191::Storage.TaskManager.Task::(prepare) Task=`75016a21-7418-41ea-82e8-5a4329aba567`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:24,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`75016a21-7418-41ea-82e8-5a4329aba567`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:50:24,626::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:50:24,627::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:50:24,627::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:50:24,627::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:50:24,628::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:50:24,628::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:50:24,628::task::993::Storage.TaskManager.Task::(_decref) Task=`75016a21-7418-41ea-82e8-5a4329aba567`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:50:25,949::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:25,949::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:25,949::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:25,950::__init__::232::IOProcess::(_processLogs) (7812) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:25,951::__init__::232::IOProcess::(_processLogs) DEBUG|(7812) Queuing response >Thread-16::DEBUG::2014-08-11 15:50:25,952::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:50:25,971::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420755 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:25,972::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:25,973::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:25,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:25,973::__init__::232::IOProcess::(_processLogs) (7813) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:25,974::__init__::232::IOProcess::(_processLogs) DEBUG|(7813) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:25,975::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:25,976::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:25,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:25,977::__init__::232::IOProcess::(_processLogs) (7814) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:25,977::__init__::232::IOProcess::(_processLogs) (7814) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:25,978::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:25,978::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:25,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:25,979::__init__::232::IOProcess::(_processLogs) (7815) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:25,979::__init__::232::IOProcess::(_processLogs) (7815) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:25,980::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:25,980::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:25,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:25,981::__init__::232::IOProcess::(_processLogs) (7816) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:25,982::__init__::232::IOProcess::(_processLogs) (7816) Queuing response >Thread-122::DEBUG::2014-08-11 15:50:26,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ea90f07-9bed-41a5-af14-5371163665db`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:50:26,150::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:50:26,151::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:26,151::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:26,151::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:26,151::__init__::232::IOProcess::(_processLogs) (7817) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:50:26,152::__init__::232::IOProcess::(_processLogs) DEBUG|(7817) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:26,153::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:26,154::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:50:26,154::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:50:26,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:26,156::__init__::232::IOProcess::(_processLogs) (7818) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:50:26,156::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ea90f07-9bed-41a5-af14-5371163665db`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:50:26,157::__init__::232::IOProcess::(_processLogs) (7818) Queuing response >Thread-122::DEBUG::2014-08-11 15:50:26,158::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ea90f07-9bed-41a5-af14-5371163665db`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:50:26,159::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:50:26,159::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:50:26,159::task::993::Storage.TaskManager.Task::(_decref) Task=`0ea90f07-9bed-41a5-af14-5371163665db`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:50:26,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:26,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131862 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:28,502::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:28,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135347 s, 75.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:29,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:29,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:29,678::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:50:30,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:30,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013363 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:32,615::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:32,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0275986 s, 37.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:50:34,640::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:50:34,641::task::595::Storage.TaskManager.Task::(_updateState) Task=`703486f6-a47c-47a7-90c6-26ef4f2c1cdb`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:34,679::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:34,686::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:34,687::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:34,688::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:50:34,692::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:34,693::task::1191::Storage.TaskManager.Task::(prepare) Task=`703486f6-a47c-47a7-90c6-26ef4f2c1cdb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:34,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`703486f6-a47c-47a7-90c6-26ef4f2c1cdb`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:34,693::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:34,693::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:34,694::task::993::Storage.TaskManager.Task::(_decref) Task=`703486f6-a47c-47a7-90c6-26ef4f2c1cdb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:50:34,704::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:50:34,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`966c57b0-aaa9-4aff-9185-cdcb0bcc512c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:50:34,705::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:50:34,706::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`09a150bc-c16b-4b1b-8052-e17fe87833b1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:50:34,706::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:50:34,706::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:50:34,706::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`09a150bc-c16b-4b1b-8052-e17fe87833b1`::Granted request >Thread-34::DEBUG::2014-08-11 15:50:34,707::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`966c57b0-aaa9-4aff-9185-cdcb0bcc512c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:50:34,707::task::993::Storage.TaskManager.Task::(_decref) Task=`966c57b0-aaa9-4aff-9185-cdcb0bcc512c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:50:34,727::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:34,728::task::1191::Storage.TaskManager.Task::(prepare) Task=`966c57b0-aaa9-4aff-9185-cdcb0bcc512c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:34,728::task::595::Storage.TaskManager.Task::(_updateState) Task=`966c57b0-aaa9-4aff-9185-cdcb0bcc512c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:50:34,728::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:50:34,728::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:50:34,729::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:50:34,729::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:50:34,729::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:50:34,729::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:50:34,730::task::993::Storage.TaskManager.Task::(_decref) Task=`966c57b0-aaa9-4aff-9185-cdcb0bcc512c`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:50:34,847::task::595::Storage.TaskManager.Task::(_updateState) Task=`2bc645a8-a3a7-41c7-bf7d-1faa434bde81`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:34,847::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:50:34,848::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420755', 'lastCheck': '8.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:34,848::task::1191::Storage.TaskManager.Task::(prepare) Task=`2bc645a8-a3a7-41c7-bf7d-1faa434bde81`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420755', 'lastCheck': '8.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:34,848::task::595::Storage.TaskManager.Task::(_updateState) Task=`2bc645a8-a3a7-41c7-bf7d-1faa434bde81`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:34,848::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:34,849::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:34,849::task::993::Storage.TaskManager.Task::(_decref) Task=`2bc645a8-a3a7-41c7-bf7d-1faa434bde81`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:50:34,867::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:50:34,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:34,966::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133029 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:35,982::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:35,982::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:35,983::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:35,983::__init__::232::IOProcess::(_processLogs) (7819) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:35,983::__init__::232::IOProcess::(_processLogs) (7819) Queuing response >Thread-16::DEBUG::2014-08-11 15:50:35,984::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:50:36,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420624 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:36,005::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:36,005::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:36,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:36,006::__init__::232::IOProcess::(_processLogs) DEBUG|(7820) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:36,007::__init__::232::IOProcess::(_processLogs) (7820) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:36,008::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:36,008::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:36,008::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:36,009::__init__::232::IOProcess::(_processLogs) (7821) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:36,009::__init__::232::IOProcess::(_processLogs) DEBUG|(7821) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:36,010::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:36,011::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:36,011::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:36,012::__init__::232::IOProcess::(_processLogs) (7822) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:36,012::__init__::232::IOProcess::(_processLogs) (7822) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:36,012::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:36,012::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:36,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:36,013::__init__::232::IOProcess::(_processLogs) (7823) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:36,013::__init__::232::IOProcess::(_processLogs) DEBUG|(7823) Queuing response >Dummy-33::DEBUG::2014-08-11 15:50:36,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:37,022::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131121 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:39,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:39,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136017 s, 75.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:39,695::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:39,696::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:39,782::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:50:41,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:41,136::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133092 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:43,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:43,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131692 s, 77.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:50:44,742::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:50:44,743::task::595::Storage.TaskManager.Task::(_updateState) Task=`37f54476-fb18-4387-90f8-8f7ed529aa89`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:44,781::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:44,789::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:44,791::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:44,792::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:50:44,795::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:44,795::task::1191::Storage.TaskManager.Task::(prepare) Task=`37f54476-fb18-4387-90f8-8f7ed529aa89`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:44,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`37f54476-fb18-4387-90f8-8f7ed529aa89`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:44,796::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:44,796::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:44,796::task::993::Storage.TaskManager.Task::(_decref) Task=`37f54476-fb18-4387-90f8-8f7ed529aa89`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:50:44,806::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:50:44,807::task::595::Storage.TaskManager.Task::(_updateState) Task=`e909cac9-ad72-4333-bca2-08225b1e8e0c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:50:44,807::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:50:44,808::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`72b50efd-8054-4193-b267-d6123306fe95`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:50:44,808::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:50:44,809::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:50:44,809::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`72b50efd-8054-4193-b267-d6123306fe95`::Granted request >Thread-34::DEBUG::2014-08-11 15:50:44,809::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e909cac9-ad72-4333-bca2-08225b1e8e0c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:50:44,810::task::993::Storage.TaskManager.Task::(_decref) Task=`e909cac9-ad72-4333-bca2-08225b1e8e0c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:50:44,829::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:44,830::task::1191::Storage.TaskManager.Task::(prepare) Task=`e909cac9-ad72-4333-bca2-08225b1e8e0c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:44,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`e909cac9-ad72-4333-bca2-08225b1e8e0c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:50:44,830::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:50:44,831::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:50:44,831::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:50:44,831::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:50:44,831::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:50:44,832::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:50:44,832::task::993::Storage.TaskManager.Task::(_decref) Task=`e909cac9-ad72-4333-bca2-08225b1e8e0c`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:50:45,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:45,248::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131038 s, 78.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:46,015::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:46,016::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:46,016::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:46,016::__init__::232::IOProcess::(_processLogs) (7824) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:46,017::__init__::232::IOProcess::(_processLogs) DEBUG|(7824) Queuing response >Thread-16::DEBUG::2014-08-11 15:50:46,018::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:50:46,038::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423752 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:46,039::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:46,040::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:46,040::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:46,040::__init__::232::IOProcess::(_processLogs) (7825) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:46,041::__init__::232::IOProcess::(_processLogs) DEBUG|(7825) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:46,042::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:46,043::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:46,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:46,044::__init__::232::IOProcess::(_processLogs) (7826) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:46,044::__init__::232::IOProcess::(_processLogs) (7826) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:46,044::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:46,045::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:46,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:46,045::__init__::232::IOProcess::(_processLogs) (7827) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:46,046::__init__::232::IOProcess::(_processLogs) DEBUG|(7827) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:46,046::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:46,047::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:46,048::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:46,048::__init__::232::IOProcess::(_processLogs) (7828) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:46,049::__init__::232::IOProcess::(_processLogs) (7828) Queuing response >Dummy-33::DEBUG::2014-08-11 15:50:47,271::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:47,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132637 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:49,327::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:49,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134855 s, 75.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:49,799::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:49,884::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:49,885::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:50:49,957::task::595::Storage.TaskManager.Task::(_updateState) Task=`99fa32e4-4c52-4953-bc34-3eb1543e7aa9`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:49,957::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:50:49,958::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423752', 'lastCheck': '3.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:49,958::task::1191::Storage.TaskManager.Task::(prepare) Task=`99fa32e4-4c52-4953-bc34-3eb1543e7aa9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423752', 'lastCheck': '3.9', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:50:49,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`99fa32e4-4c52-4953-bc34-3eb1543e7aa9`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:49,958::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:49,958::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:49,959::task::993::Storage.TaskManager.Task::(_decref) Task=`99fa32e4-4c52-4953-bc34-3eb1543e7aa9`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:50:49,978::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:50:51,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:51,417::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133352 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:53,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:53,473::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132006 s, 77.6 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:50:54,843::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:50:54,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`38bd77e7-161f-4c26-89d2-59052bbdae90`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:50:54,883::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:54,893::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:54,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:54,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:50:54,896::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:54,897::task::1191::Storage.TaskManager.Task::(prepare) Task=`38bd77e7-161f-4c26-89d2-59052bbdae90`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:50:54,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`38bd77e7-161f-4c26-89d2-59052bbdae90`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:50:54,897::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:50:54,897::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:50:54,898::task::993::Storage.TaskManager.Task::(_decref) Task=`38bd77e7-161f-4c26-89d2-59052bbdae90`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:50:54,908::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:50:54,909::task::595::Storage.TaskManager.Task::(_updateState) Task=`81249304-a0cd-43c2-9484-df3948261a66`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:50:54,909::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:50:54,910::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f8bb7eb-6578-47eb-bb6c-7c7959d7e0ef`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:50:54,910::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:50:54,910::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:50:54,911::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9f8bb7eb-6578-47eb-bb6c-7c7959d7e0ef`::Granted request >Thread-34::DEBUG::2014-08-11 15:50:54,911::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`81249304-a0cd-43c2-9484-df3948261a66`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:50:54,912::task::993::Storage.TaskManager.Task::(_decref) Task=`81249304-a0cd-43c2-9484-df3948261a66`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:50:54,931::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:54,932::task::1191::Storage.TaskManager.Task::(prepare) Task=`81249304-a0cd-43c2-9484-df3948261a66`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:50:54,932::task::595::Storage.TaskManager.Task::(_updateState) Task=`81249304-a0cd-43c2-9484-df3948261a66`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:50:54,932::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:50:54,932::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:50:54,933::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:50:54,933::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:50:54,933::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:50:54,934::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:50:54,934::task::993::Storage.TaskManager.Task::(_decref) Task=`81249304-a0cd-43c2-9484-df3948261a66`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:50:55,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:55,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015536 s, 65.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:56,049::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:56,050::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:50:56,051::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:50:56,051::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:56,070::__init__::232::IOProcess::(_processLogs) (7829) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:56,071::__init__::232::IOProcess::(_processLogs) (7829) Queuing response >Thread-16::DEBUG::2014-08-11 15:50:56,072::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000654486 s, 695 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:50:56,073::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:56,073::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:56,074::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:56,074::__init__::232::IOProcess::(_processLogs) (7830) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:50:56,075::__init__::232::IOProcess::(_processLogs) DEBUG|(7830) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:56,076::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:56,077::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:56,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:56,078::__init__::232::IOProcess::(_processLogs) (7831) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:56,078::__init__::232::IOProcess::(_processLogs) (7831) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:56,078::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:56,079::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:56,079::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:56,080::__init__::232::IOProcess::(_processLogs) (7832) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:56,080::__init__::232::IOProcess::(_processLogs) (7832) Queuing response >Thread-14::DEBUG::2014-08-11 15:50:56,080::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:50:56,081::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:50:56,082::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:50:56,082::__init__::232::IOProcess::(_processLogs) (7833) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:50:56,083::__init__::232::IOProcess::(_processLogs) (7833) Queuing response >Dummy-33::DEBUG::2014-08-11 15:50:57,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:57,591::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174726 s, 58.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:50:59,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:50:59,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131489 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:59,903::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:59,904::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:50:59,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:51:01,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:01,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133471 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:03,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:03,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013223 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:04,909::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:04,910::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:51:04,945::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:51:04,986::task::595::Storage.TaskManager.Task::(_updateState) Task=`0107670e-0e63-436c-84ed-bb21f057056f`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:04,988::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:51:04,988::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-128::INFO::2014-08-11 15:51:05,002::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:05,003::task::1191::Storage.TaskManager.Task::(prepare) Task=`0107670e-0e63-436c-84ed-bb21f057056f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:05,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`0107670e-0e63-436c-84ed-bb21f057056f`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:05,003::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:05,003::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:05,004::task::993::Storage.TaskManager.Task::(_decref) Task=`0107670e-0e63-436c-84ed-bb21f057056f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:51:05,016::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:51:05,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`df47f8ef-aed8-4a69-b276-05539d7555e8`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:51:05,016::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:51:05,017::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`265039de-f181-42ff-96db-3cf6c9351144`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:51:05,017::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:51:05,018::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:51:05,018::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`265039de-f181-42ff-96db-3cf6c9351144`::Granted request >Thread-34::DEBUG::2014-08-11 15:51:05,019::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`df47f8ef-aed8-4a69-b276-05539d7555e8`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:51:05,019::task::993::Storage.TaskManager.Task::(_decref) Task=`df47f8ef-aed8-4a69-b276-05539d7555e8`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:51:05,039::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:05,039::task::1191::Storage.TaskManager.Task::(prepare) Task=`df47f8ef-aed8-4a69-b276-05539d7555e8`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:05,039::task::595::Storage.TaskManager.Task::(_updateState) Task=`df47f8ef-aed8-4a69-b276-05539d7555e8`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:51:05,040::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:51:05,040::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:51:05,040::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:51:05,040::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:51:05,041::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:51:05,041::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:51:05,041::task::993::Storage.TaskManager.Task::(_decref) Task=`df47f8ef-aed8-4a69-b276-05539d7555e8`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:51:05,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`a9e9f78e-2b61-497e-8c15-208b2b1f5b8b`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:05,075::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:51:05,075::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000654486', 'lastCheck': '9.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:05,075::task::1191::Storage.TaskManager.Task::(prepare) Task=`a9e9f78e-2b61-497e-8c15-208b2b1f5b8b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000654486', 'lastCheck': '9.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:05,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`a9e9f78e-2b61-497e-8c15-208b2b1f5b8b`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:05,076::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:05,076::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:05,076::task::993::Storage.TaskManager.Task::(_decref) Task=`a9e9f78e-2b61-497e-8c15-208b2b1f5b8b`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:51:05,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:51:05,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:05,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132831 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:06,083::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:06,083::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:06,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:06,084::__init__::232::IOProcess::(_processLogs) (7834) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:06,084::__init__::232::IOProcess::(_processLogs) (7834) Queuing response >Thread-16::DEBUG::2014-08-11 15:51:06,085::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:51:06,105::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041336 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:06,106::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:06,106::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:06,107::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:06,107::__init__::232::IOProcess::(_processLogs) (7835) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:06,107::__init__::232::IOProcess::(_processLogs) (7835) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:06,108::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:06,109::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:06,109::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:06,110::__init__::232::IOProcess::(_processLogs) (7836) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:06,110::__init__::232::IOProcess::(_processLogs) (7836) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:06,110::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:06,111::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:06,112::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:06,112::__init__::232::IOProcess::(_processLogs) (7837) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:06,112::__init__::232::IOProcess::(_processLogs) (7837) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:06,112::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:06,113::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:06,114::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:06,115::__init__::232::IOProcess::(_processLogs) (7838) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:06,115::__init__::232::IOProcess::(_processLogs) (7838) Queuing response >Dummy-33::DEBUG::2014-08-11 15:51:07,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:07,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131181 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:09,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:09,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131627 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:09,994::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:09,996::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:09,997::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:51:11,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:11,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133612 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:14,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:14,047::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01345 s, 76.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:15,004::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:15,005::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:51:15,056::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:51:15,087::task::595::Storage.TaskManager.Task::(_updateState) Task=`87d748f9-42c8-4127-8cce-55be6e9f45a6`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:15,089::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:15,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:51:15,103::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:15,103::task::1191::Storage.TaskManager.Task::(prepare) Task=`87d748f9-42c8-4127-8cce-55be6e9f45a6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:15,103::task::595::Storage.TaskManager.Task::(_updateState) Task=`87d748f9-42c8-4127-8cce-55be6e9f45a6`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:15,104::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:15,104::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:15,104::task::993::Storage.TaskManager.Task::(_decref) Task=`87d748f9-42c8-4127-8cce-55be6e9f45a6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:51:15,114::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:51:15,115::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3e2de63-5ab3-4cd3-9381-da8d797d1649`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:51:15,115::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:51:15,116::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`20077439-aba9-4d2c-bb5f-99c7a9a38fd4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:51:15,116::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:51:15,117::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:51:15,117::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`20077439-aba9-4d2c-bb5f-99c7a9a38fd4`::Granted request >Thread-34::DEBUG::2014-08-11 15:51:15,117::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c3e2de63-5ab3-4cd3-9381-da8d797d1649`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:51:15,118::task::993::Storage.TaskManager.Task::(_decref) Task=`c3e2de63-5ab3-4cd3-9381-da8d797d1649`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:51:15,137::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951822336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:15,138::task::1191::Storage.TaskManager.Task::(prepare) Task=`c3e2de63-5ab3-4cd3-9381-da8d797d1649`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951822336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:15,138::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3e2de63-5ab3-4cd3-9381-da8d797d1649`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:51:15,138::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:51:15,138::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:51:15,139::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:51:15,139::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:51:15,139::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:51:15,140::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:51:15,140::task::993::Storage.TaskManager.Task::(_decref) Task=`c3e2de63-5ab3-4cd3-9381-da8d797d1649`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:51:16,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:16,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131223 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:16,116::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:16,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:16,118::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:16,120::__init__::232::IOProcess::(_processLogs) DEBUG|(7839) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:51:16,122::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:51:16,122::__init__::232::IOProcess::(_processLogs) (7839) Queuing response >Thread-16::DEBUG::2014-08-11 15:51:16,145::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000724856 s, 628 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:16,147::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:16,149::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:16,150::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:16,151::__init__::232::IOProcess::(_processLogs) (7840) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:16,153::__init__::232::IOProcess::(_processLogs) DEBUG|(7840) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:16,155::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:16,156::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:16,157::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:16,158::__init__::232::IOProcess::(_processLogs) (7841) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:16,160::__init__::232::IOProcess::(_processLogs) DEBUG|(7841) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:16,162::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:16,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:16,164::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:16,165::__init__::232::IOProcess::(_processLogs) (7842) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:16,167::__init__::232::IOProcess::(_processLogs) DEBUG|(7842) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:16,169::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:16,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:16,171::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:16,172::__init__::232::IOProcess::(_processLogs) (7843) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:16,174::__init__::232::IOProcess::(_processLogs) DEBUG|(7843) Queuing response >Dummy-33::DEBUG::2014-08-11 15:51:18,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:18,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133313 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:20,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:20,097::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:20,188::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:51:20,191::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d21cd27-4cc3-4da5-a83c-c749c87fd16c`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:20,192::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:51:20,192::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000724856', 'lastCheck': '4.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:20,192::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d21cd27-4cc3-4da5-a83c-c749c87fd16c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000724856', 'lastCheck': '4.0', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:20,193::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d21cd27-4cc3-4da5-a83c-c749c87fd16c`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:20,193::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:20,193::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:20,193::task::993::Storage.TaskManager.Task::(_decref) Task=`6d21cd27-4cc3-4da5-a83c-c749c87fd16c`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:51:20,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:51:20,239::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:20,273::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133158 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:22,296::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:22,330::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132974 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:24,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:24,387::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131419 s, 77.9 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:51:25,151::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:51:25,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce43047b-7062-4360-8aeb-fee6f0b0a7d7`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:25,184::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:25,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:51:25,198::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:25,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:51:25,200::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce43047b-7062-4360-8aeb-fee6f0b0a7d7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:25,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce43047b-7062-4360-8aeb-fee6f0b0a7d7`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:25,202::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:25,203::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:25,205::task::993::Storage.TaskManager.Task::(_decref) Task=`ce43047b-7062-4360-8aeb-fee6f0b0a7d7`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:25,205::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:51:25,217::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:51:25,217::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7b1b181-a32d-4e75-a742-81176d85c5b0`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:51:25,218::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:51:25,218::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a92ce773-b08e-47b4-9c4f-86e069cc0495`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:51:25,219::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:51:25,219::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:51:25,219::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a92ce773-b08e-47b4-9c4f-86e069cc0495`::Granted request >Thread-34::DEBUG::2014-08-11 15:51:25,220::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c7b1b181-a32d-4e75-a742-81176d85c5b0`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:51:25,220::task::993::Storage.TaskManager.Task::(_decref) Task=`c7b1b181-a32d-4e75-a742-81176d85c5b0`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:51:25,240::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:25,240::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7b1b181-a32d-4e75-a742-81176d85c5b0`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:25,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7b1b181-a32d-4e75-a742-81176d85c5b0`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:51:25,241::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:51:25,241::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:51:25,241::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:51:25,242::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:51:25,242::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:51:25,242::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:51:25,243::task::993::Storage.TaskManager.Task::(_decref) Task=`c7b1b181-a32d-4e75-a742-81176d85c5b0`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:51:26,176::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:26,177::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:26,177::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:26,177::__init__::232::IOProcess::(_processLogs) (7844) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:26,178::__init__::232::IOProcess::(_processLogs) DEBUG|(7844) Queuing response >Thread-16::DEBUG::2014-08-11 15:51:26,179::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:51:26,198::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432436 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:26,200::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:26,200::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:26,200::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:26,201::__init__::232::IOProcess::(_processLogs) (7845) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:26,201::__init__::232::IOProcess::(_processLogs) DEBUG|(7845) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:26,203::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:26,203::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:26,204::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:26,204::__init__::232::IOProcess::(_processLogs) (7846) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:26,204::__init__::232::IOProcess::(_processLogs) (7846) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:26,205::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:26,205::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:26,206::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:26,206::__init__::232::IOProcess::(_processLogs) (7847) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:26,207::__init__::232::IOProcess::(_processLogs) (7847) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:26,207::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:26,208::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:26,209::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:26,209::__init__::232::IOProcess::(_processLogs) (7848) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:26,209::__init__::232::IOProcess::(_processLogs) (7848) Queuing response >Thread-122::DEBUG::2014-08-11 15:51:26,262::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5f8e23c-72db-42eb-93a4-f58d40500903`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:51:26,263::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:51:26,263::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:26,264::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:26,264::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:26,264::__init__::232::IOProcess::(_processLogs) (7849) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:51:26,265::__init__::232::IOProcess::(_processLogs) DEBUG|(7849) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:26,266::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:26,267::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:51:26,267::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:51:26,268::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:51:26,269::task::1191::Storage.TaskManager.Task::(prepare) Task=`e5f8e23c-72db-42eb-93a4-f58d40500903`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:51:26,269::__init__::232::IOProcess::(_processLogs) (7850) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:51:26,270::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5f8e23c-72db-42eb-93a4-f58d40500903`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:51:26,271::__init__::232::IOProcess::(_processLogs) (7850) Queuing response >Thread-122::DEBUG::2014-08-11 15:51:26,271::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:51:26,272::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:51:26,272::task::993::Storage.TaskManager.Task::(_decref) Task=`e5f8e23c-72db-42eb-93a4-f58d40500903`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:51:26,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:26,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132454 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:28,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:28,501::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133135 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:30,214::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:30,215::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:30,285::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:51:30,524::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:30,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134505 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:32,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:32,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.027131 s, 37.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:34,879::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:34,913::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133155 s, 76.9 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:51:35,255::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:51:35,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`303ef60f-7314-441d-9add-a0a0475a01c1`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:35,284::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:35,293::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:35,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:35,295::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:51:35,297::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:35,298::task::1191::Storage.TaskManager.Task::(prepare) Task=`303ef60f-7314-441d-9add-a0a0475a01c1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:35,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`303ef60f-7314-441d-9add-a0a0475a01c1`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:35,298::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:35,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:35,299::task::993::Storage.TaskManager.Task::(_decref) Task=`303ef60f-7314-441d-9add-a0a0475a01c1`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:51:35,312::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:51:35,313::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5612003-e62c-4524-aea2-3600932578b3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:51:35,313::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:51:35,314::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b3480a56-2142-4ede-aa32-22da98698222`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:51:35,314::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:51:35,314::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:51:35,315::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b3480a56-2142-4ede-aa32-22da98698222`::Granted request >Thread-34::DEBUG::2014-08-11 15:51:35,315::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5612003-e62c-4524-aea2-3600932578b3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:51:35,315::task::993::Storage.TaskManager.Task::(_decref) Task=`d5612003-e62c-4524-aea2-3600932578b3`::ref 1 aborting False >Thread-128::DEBUG::2014-08-11 15:51:35,329::task::595::Storage.TaskManager.Task::(_updateState) Task=`91bbd3ba-14ea-49ea-9c89-2bf080392020`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:35,331::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-128::INFO::2014-08-11 15:51:35,334::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432436', 'lastCheck': '9.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:35,335::task::1191::Storage.TaskManager.Task::(prepare) Task=`91bbd3ba-14ea-49ea-9c89-2bf080392020`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432436', 'lastCheck': '9.1', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:35,337::task::595::Storage.TaskManager.Task::(_updateState) Task=`91bbd3ba-14ea-49ea-9c89-2bf080392020`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:35,337::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:35,337::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:35,338::task::993::Storage.TaskManager.Task::(_decref) Task=`91bbd3ba-14ea-49ea-9c89-2bf080392020`::ref 0 aborting False >Thread-34::INFO::2014-08-11 15:51:35,349::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:35,350::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5612003-e62c-4524-aea2-3600932578b3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:35,350::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5612003-e62c-4524-aea2-3600932578b3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:51:35,350::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:51:35,350::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:51:35,351::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:51:35,351::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:51:35,351::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:51:35,352::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:51:35,352::task::993::Storage.TaskManager.Task::(_decref) Task=`d5612003-e62c-4524-aea2-3600932578b3`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:51:35,360::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:51:36,209::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:36,210::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:36,210::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:36,210::__init__::232::IOProcess::(_processLogs) (7851) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:36,211::__init__::232::IOProcess::(_processLogs) DEBUG|(7851) Queuing response >Thread-16::DEBUG::2014-08-11 15:51:36,212::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:51:36,232::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421407 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:36,233::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:36,234::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:36,234::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:36,234::__init__::232::IOProcess::(_processLogs) (7852) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:36,235::__init__::232::IOProcess::(_processLogs) DEBUG|(7852) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:36,236::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:36,236::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:36,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:36,237::__init__::232::IOProcess::(_processLogs) (7853) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:36,237::__init__::232::IOProcess::(_processLogs) DEBUG|(7853) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:36,238::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:36,239::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:36,240::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:36,240::__init__::232::IOProcess::(_processLogs) (7854) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:36,240::__init__::232::IOProcess::(_processLogs) (7854) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:36,240::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:36,241::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:36,241::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:36,241::__init__::232::IOProcess::(_processLogs) (7855) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:36,242::__init__::232::IOProcess::(_processLogs) DEBUG|(7855) Queuing response >Dummy-33::DEBUG::2014-08-11 15:51:36,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:36,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132334 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:38,991::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:39,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132583 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:40,302::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:40,304::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:40,385::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:51:41,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:41,082::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133039 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:43,105::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:43,139::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131647 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:45,161::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:45,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133212 s, 76.9 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:51:45,364::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:51:45,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`54f6bfbf-8713-4932-9a11-0ee897e117af`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:45,384::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:45,393::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:45,394::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:45,395::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:51:45,398::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:45,398::task::1191::Storage.TaskManager.Task::(prepare) Task=`54f6bfbf-8713-4932-9a11-0ee897e117af`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:45,399::task::595::Storage.TaskManager.Task::(_updateState) Task=`54f6bfbf-8713-4932-9a11-0ee897e117af`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:45,399::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:45,399::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:45,399::task::993::Storage.TaskManager.Task::(_decref) Task=`54f6bfbf-8713-4932-9a11-0ee897e117af`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:51:45,409::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:51:45,410::task::595::Storage.TaskManager.Task::(_updateState) Task=`967be284-93fb-4a57-af87-ed1d575019c7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:51:45,410::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:51:45,411::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0a71c3c6-4eb9-4f41-8404-d33400783065`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:51:45,411::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:51:45,412::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:51:45,412::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0a71c3c6-4eb9-4f41-8404-d33400783065`::Granted request >Thread-34::DEBUG::2014-08-11 15:51:45,413::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`967be284-93fb-4a57-af87-ed1d575019c7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:51:45,413::task::993::Storage.TaskManager.Task::(_decref) Task=`967be284-93fb-4a57-af87-ed1d575019c7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:51:45,433::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:45,433::task::1191::Storage.TaskManager.Task::(prepare) Task=`967be284-93fb-4a57-af87-ed1d575019c7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:45,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`967be284-93fb-4a57-af87-ed1d575019c7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:51:45,434::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:51:45,434::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:51:45,434::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:51:45,434::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:51:45,435::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:51:45,435::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:51:45,435::task::993::Storage.TaskManager.Task::(_decref) Task=`967be284-93fb-4a57-af87-ed1d575019c7`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:51:46,243::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:46,244::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:46,244::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:46,244::__init__::232::IOProcess::(_processLogs) (7856) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:46,245::__init__::232::IOProcess::(_processLogs) (7856) Queuing response >Thread-16::DEBUG::2014-08-11 15:51:46,245::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:51:46,265::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423612 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:46,266::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:46,266::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:46,267::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:46,267::__init__::232::IOProcess::(_processLogs) (7857) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:46,267::__init__::232::IOProcess::(_processLogs) (7857) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:46,268::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:46,269::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:46,269::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:46,270::__init__::232::IOProcess::(_processLogs) (7858) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:46,270::__init__::232::IOProcess::(_processLogs) (7858) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:46,270::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:46,270::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:46,271::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:46,271::__init__::232::IOProcess::(_processLogs) (7859) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:46,271::__init__::232::IOProcess::(_processLogs) DEBUG|(7859) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:46,272::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:46,273::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:46,273::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:46,274::__init__::232::IOProcess::(_processLogs) (7860) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:46,275::__init__::232::IOProcess::(_processLogs) (7860) Queuing response >Dummy-33::DEBUG::2014-08-11 15:51:47,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:47,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132177 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:49,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:49,310::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148903 s, 68.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:50,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:50,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::DEBUG::2014-08-11 15:51:50,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`52686603-44c0-444b-92f6-862d256e1b98`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:50,486::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:50,488::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:51:50,489::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423612', 'lastCheck': '4.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:50,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`52686603-44c0-444b-92f6-862d256e1b98`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423612', 'lastCheck': '4.2', 'valid': True}} >Thread-128::DEBUG::2014-08-11 15:51:50,492::task::595::Storage.TaskManager.Task::(_updateState) Task=`52686603-44c0-444b-92f6-862d256e1b98`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:50,492::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:50,493::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:50,493::task::993::Storage.TaskManager.Task::(_decref) Task=`52686603-44c0-444b-92f6-862d256e1b98`::ref 0 aborting False >Thread-128::DEBUG::2014-08-11 15:51:50,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:51:51,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:51,367::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132598 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:53,390::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:53,423::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133942 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:55,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-128::DEBUG::2014-08-11 15:51:55,466::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Dummy-33::DEBUG::2014-08-11 15:51:55,480::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136844 s, 74.8 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:51:55,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`7236125c-bddd-48ae-8161-cd842e9ee24e`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:51:55,488::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:55,498::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:55,499::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:51:55,502::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:51:55,504::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:55,506::task::1191::Storage.TaskManager.Task::(prepare) Task=`7236125c-bddd-48ae-8161-cd842e9ee24e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:51:55,508::task::595::Storage.TaskManager.Task::(_updateState) Task=`7236125c-bddd-48ae-8161-cd842e9ee24e`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:51:55,510::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:51:55,510::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:51:55,511::task::993::Storage.TaskManager.Task::(_decref) Task=`7236125c-bddd-48ae-8161-cd842e9ee24e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:51:55,525::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:51:55,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d3eb86b-7803-4f9f-9f11-bfdf1abc72e9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:51:55,528::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:51:55,533::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d81ca3fe-827c-4716-90f6-5f7a70200938`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:51:55,534::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:51:55,535::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:51:55,536::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d81ca3fe-827c-4716-90f6-5f7a70200938`::Granted request >Thread-34::DEBUG::2014-08-11 15:51:55,538::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7d3eb86b-7803-4f9f-9f11-bfdf1abc72e9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:51:55,539::task::993::Storage.TaskManager.Task::(_decref) Task=`7d3eb86b-7803-4f9f-9f11-bfdf1abc72e9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:51:55,565::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:55,565::task::1191::Storage.TaskManager.Task::(prepare) Task=`7d3eb86b-7803-4f9f-9f11-bfdf1abc72e9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:51:55,565::task::595::Storage.TaskManager.Task::(_updateState) Task=`7d3eb86b-7803-4f9f-9f11-bfdf1abc72e9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:51:55,565::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:51:55,566::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:51:55,566::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:51:55,566::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:51:55,566::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:51:55,567::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:51:55,567::task::993::Storage.TaskManager.Task::(_decref) Task=`7d3eb86b-7803-4f9f-9f11-bfdf1abc72e9`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:51:56,275::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:56,275::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:56,276::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:56,276::__init__::232::IOProcess::(_processLogs) (7861) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:56,276::__init__::232::IOProcess::(_processLogs) (7861) Queuing response >Thread-16::DEBUG::2014-08-11 15:51:56,277::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:51:56,295::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416679 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:51:56,297::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:56,297::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:56,297::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:56,297::__init__::232::IOProcess::(_processLogs) (7862) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:51:56,298::__init__::232::IOProcess::(_processLogs) DEBUG|(7862) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:56,299::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:56,300::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:56,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:56,301::__init__::232::IOProcess::(_processLogs) (7863) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:56,301::__init__::232::IOProcess::(_processLogs) (7863) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:56,302::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:56,302::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:56,303::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:56,303::__init__::232::IOProcess::(_processLogs) (7864) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:56,303::__init__::232::IOProcess::(_processLogs) (7864) Queuing response >Thread-14::DEBUG::2014-08-11 15:51:56,304::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:51:56,304::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:51:56,305::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:51:56,306::__init__::232::IOProcess::(_processLogs) (7865) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:51:56,306::__init__::232::IOProcess::(_processLogs) (7865) Queuing response >Dummy-33::DEBUG::2014-08-11 15:51:57,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:57,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131909 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:51:59,604::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:51:59,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131851 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:00,510::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:00,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:00,585::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:52:01,660::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:01,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153279 s, 66.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:03,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:03,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132753 s, 77.1 MB/s\n'; <rc> = 0 >Thread-128::DEBUG::2014-08-11 15:52:05,579::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-128::DEBUG::2014-08-11 15:52:05,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`b27d97f3-836a-4126-8505-77789a5a798c`::moving from state init -> state preparing >Thread-128::INFO::2014-08-11 15:52:05,584::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:05,593::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:05,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:05,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-128::INFO::2014-08-11 15:52:05,597::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:52:05,597::task::1191::Storage.TaskManager.Task::(prepare) Task=`b27d97f3-836a-4126-8505-77789a5a798c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-128::DEBUG::2014-08-11 15:52:05,598::task::595::Storage.TaskManager.Task::(_updateState) Task=`b27d97f3-836a-4126-8505-77789a5a798c`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-08-11 15:52:05,598::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-08-11 15:52:05,598::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-08-11 15:52:05,598::task::993::Storage.TaskManager.Task::(_decref) Task=`b27d97f3-836a-4126-8505-77789a5a798c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:52:05,615::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:52:05,616::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7a3f779-8659-414b-b815-b6a2243a7ac3`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:52:05,616::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:52:05,617::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dc4e179d-1141-4eb4-bd3c-e5c6606e8ab5`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:52:05,617::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:52:05,617::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:52:05,617::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dc4e179d-1141-4eb4-bd3c-e5c6606e8ab5`::Granted request >Thread-34::DEBUG::2014-08-11 15:52:05,618::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d7a3f779-8659-414b-b815-b6a2243a7ac3`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:52:05,618::task::993::Storage.TaskManager.Task::(_decref) Task=`d7a3f779-8659-414b-b815-b6a2243a7ac3`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:52:05,639::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:05,639::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7a3f779-8659-414b-b815-b6a2243a7ac3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:05,639::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7a3f779-8659-414b-b815-b6a2243a7ac3`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:52:05,640::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:52:05,640::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:52:05,641::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:52:05,642::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:52:05,642::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:52:05,642::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:52:05,642::task::993::Storage.TaskManager.Task::(_decref) Task=`d7a3f779-8659-414b-b815-b6a2243a7ac3`::ref 0 aborting False >Detector thread::DEBUG::2014-08-11 15:52:05,828::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:51752 >Detector thread::DEBUG::2014-08-11 15:52:05,839::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:51752 >Detector thread::DEBUG::2014-08-11 15:52:05,840::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:51752 >Detector thread::DEBUG::2014-08-11 15:52:05,840::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 51752) >Thread-130::DEBUG::2014-08-11 15:52:05,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bdfadf9-40be-4d27-b994-6f13f62fe9cf`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:05,843::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:52:05,844::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416679', 'lastCheck': '9.5', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:05,844::task::1191::Storage.TaskManager.Task::(prepare) Task=`0bdfadf9-40be-4d27-b994-6f13f62fe9cf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416679', 'lastCheck': '9.5', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:05,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`0bdfadf9-40be-4d27-b994-6f13f62fe9cf`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:05,844::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:05,844::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:05,845::task::993::Storage.TaskManager.Task::(_decref) Task=`0bdfadf9-40be-4d27-b994-6f13f62fe9cf`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:52:05,870::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:52:05,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:06,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131751 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:06,306::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:06,307::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:06,307::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:06,307::__init__::232::IOProcess::(_processLogs) (7866) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:06,308::__init__::232::IOProcess::(_processLogs) (7866) Queuing response >Thread-16::DEBUG::2014-08-11 15:52:06,308::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:52:06,328::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414571 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:06,329::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:06,330::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:06,330::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:06,330::__init__::232::IOProcess::(_processLogs) (7867) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:06,331::__init__::232::IOProcess::(_processLogs) DEBUG|(7867) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:06,332::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:06,333::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:06,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:06,334::__init__::232::IOProcess::(_processLogs) (7868) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:06,334::__init__::232::IOProcess::(_processLogs) (7868) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:06,334::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:06,335::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:06,336::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:06,336::__init__::232::IOProcess::(_processLogs) (7869) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:06,336::__init__::232::IOProcess::(_processLogs) (7869) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:06,336::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:06,337::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:06,338::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:06,338::__init__::232::IOProcess::(_processLogs) (7870) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:06,339::__init__::232::IOProcess::(_processLogs) (7870) Queuing response >Dummy-33::DEBUG::2014-08-11 15:52:08,036::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:08,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131904 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:10,095::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:10,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0199351 s, 51.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:10,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:10,603::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:10,685::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:52:12,167::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:12,201::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132785 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:14,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:14,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132738 s, 77.1 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:52:15,664::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-130::DEBUG::2014-08-11 15:52:15,665::task::595::Storage.TaskManager.Task::(_updateState) Task=`b855b489-8d24-4f87-8b26-7617d79aaebb`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:15,684::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:15,693::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:15,694::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:15,696::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:52:15,697::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:15,697::task::1191::Storage.TaskManager.Task::(prepare) Task=`b855b489-8d24-4f87-8b26-7617d79aaebb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:15,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`b855b489-8d24-4f87-8b26-7617d79aaebb`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:15,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:15,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:15,698::task::993::Storage.TaskManager.Task::(_decref) Task=`b855b489-8d24-4f87-8b26-7617d79aaebb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:52:15,709::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [229042a1] >Thread-34::DEBUG::2014-08-11 15:52:15,709::task::595::Storage.TaskManager.Task::(_updateState) Task=`391cf655-0a56-4991-9bc2-77e76f92348e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:52:15,710::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:52:15,710::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`54cc086f-2ae6-4938-a3ff-2a62f3bffe33`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:52:15,711::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:52:15,711::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:52:15,711::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`54cc086f-2ae6-4938-a3ff-2a62f3bffe33`::Granted request >Thread-34::DEBUG::2014-08-11 15:52:15,712::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`391cf655-0a56-4991-9bc2-77e76f92348e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:52:15,712::task::993::Storage.TaskManager.Task::(_decref) Task=`391cf655-0a56-4991-9bc2-77e76f92348e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:52:15,732::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:15,732::task::1191::Storage.TaskManager.Task::(prepare) Task=`391cf655-0a56-4991-9bc2-77e76f92348e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:15,732::task::595::Storage.TaskManager.Task::(_updateState) Task=`391cf655-0a56-4991-9bc2-77e76f92348e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:52:15,733::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:52:15,733::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:52:15,733::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:52:15,734::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:52:15,734::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:52:15,734::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:52:15,734::task::993::Storage.TaskManager.Task::(_decref) Task=`391cf655-0a56-4991-9bc2-77e76f92348e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:52:16,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:16,314::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132559 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:16,339::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:16,340::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:16,340::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:16,340::__init__::232::IOProcess::(_processLogs) (7871) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:16,340::__init__::232::IOProcess::(_processLogs) (7871) Queuing response >Thread-16::DEBUG::2014-08-11 15:52:16,341::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:52:16,361::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419217 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:16,362::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:16,363::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:16,363::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:16,363::__init__::232::IOProcess::(_processLogs) (7872) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:16,363::__init__::232::IOProcess::(_processLogs) (7872) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:16,365::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:16,365::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:16,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:16,366::__init__::232::IOProcess::(_processLogs) (7873) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:16,366::__init__::232::IOProcess::(_processLogs) (7873) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:16,367::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:16,367::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:16,368::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:16,368::__init__::232::IOProcess::(_processLogs) (7874) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:16,368::__init__::232::IOProcess::(_processLogs) (7874) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:16,369::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:16,369::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:16,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:16,371::__init__::232::IOProcess::(_processLogs) (7875) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:16,371::__init__::232::IOProcess::(_processLogs) (7875) Queuing response >Dummy-33::DEBUG::2014-08-11 15:52:18,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:18,370::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133938 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:20,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:20,428::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133469 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:20,703::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:20,704::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:20,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:52:20,971::task::595::Storage.TaskManager.Task::(_updateState) Task=`438e0154-729d-4f91-b07c-5b21e58aab0e`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:20,971::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:52:20,972::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419217', 'lastCheck': '4.6', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:20,972::task::1191::Storage.TaskManager.Task::(prepare) Task=`438e0154-729d-4f91-b07c-5b21e58aab0e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419217', 'lastCheck': '4.6', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:20,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`438e0154-729d-4f91-b07c-5b21e58aab0e`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:20,972::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:20,973::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:20,973::task::993::Storage.TaskManager.Task::(_decref) Task=`438e0154-729d-4f91-b07c-5b21e58aab0e`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:52:20,992::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:52:22,450::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:22,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131916 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:24,507::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:24,541::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013138 s, 77.9 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:52:25,746::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:52:25,747::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4a817fe-8da4-494c-89b1-5b5b0818c34f`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:25,784::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:25,793::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:25,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:25,795::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:52:25,797::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:25,797::task::1191::Storage.TaskManager.Task::(prepare) Task=`f4a817fe-8da4-494c-89b1-5b5b0818c34f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:25,798::task::595::Storage.TaskManager.Task::(_updateState) Task=`f4a817fe-8da4-494c-89b1-5b5b0818c34f`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:25,798::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:25,798::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:25,798::task::993::Storage.TaskManager.Task::(_decref) Task=`f4a817fe-8da4-494c-89b1-5b5b0818c34f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:52:25,810::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:52:25,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`c806d069-c767-4bc2-bb32-b2b55045aedb`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:52:25,811::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:52:25,811::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f7899c39-bc32-432a-97d2-b9f94b32ffdf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:52:25,812::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:52:25,812::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:52:25,812::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f7899c39-bc32-432a-97d2-b9f94b32ffdf`::Granted request >Thread-34::DEBUG::2014-08-11 15:52:25,813::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c806d069-c767-4bc2-bb32-b2b55045aedb`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:52:25,813::task::993::Storage.TaskManager.Task::(_decref) Task=`c806d069-c767-4bc2-bb32-b2b55045aedb`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:52:25,833::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:25,833::task::1191::Storage.TaskManager.Task::(prepare) Task=`c806d069-c767-4bc2-bb32-b2b55045aedb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:25,834::task::595::Storage.TaskManager.Task::(_updateState) Task=`c806d069-c767-4bc2-bb32-b2b55045aedb`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:52:25,834::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:52:25,834::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:52:25,835::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:52:25,835::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:52:25,835::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:52:25,835::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:52:25,836::task::993::Storage.TaskManager.Task::(_decref) Task=`c806d069-c767-4bc2-bb32-b2b55045aedb`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:52:26,371::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:26,372::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:26,372::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:26,372::__init__::232::IOProcess::(_processLogs) (7876) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:26,373::__init__::232::IOProcess::(_processLogs) DEBUG|(7876) Queuing response >Thread-16::DEBUG::2014-08-11 15:52:26,374::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-122::DEBUG::2014-08-11 15:52:26,394::task::595::Storage.TaskManager.Task::(_updateState) Task=`5db48345-288e-4814-9daf-90e20c978caa`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:52:26,395::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-16::DEBUG::2014-08-11 15:52:26,396::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000456691 s, 996 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:26,398::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:26,399::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:26,399::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:26,399::__init__::232::IOProcess::(_processLogs) DEBUG|(7877) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:52:26,399::__init__::232::IOProcess::(_processLogs) (7877) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:26,400::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:26,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:26,402::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:26,403::__init__::232::IOProcess::(_processLogs) (7878) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:26,403::__init__::232::IOProcess::(_processLogs) (7878) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:26,403::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-14::DEBUG::2014-08-11 15:52:26,404::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:26,404::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:26,404::__init__::232::IOProcess::(_processLogs) (7879) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:52:26,404::__init__::232::IOProcess::(_processLogs) (7879) Queuing response >Thread-122::INFO::2014-08-11 15:52:26,405::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:52:26,406::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-122::DEBUG::2014-08-11 15:52:26,407::task::1191::Storage.TaskManager.Task::(prepare) Task=`5db48345-288e-4814-9daf-90e20c978caa`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:52:26,407::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:52:26,408::task::595::Storage.TaskManager.Task::(_updateState) Task=`5db48345-288e-4814-9daf-90e20c978caa`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:52:26,409::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:52:26,409::__init__::232::IOProcess::(_processLogs) (7880) Got request for method 'access' >Thread-122::DEBUG::2014-08-11 15:52:26,409::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:52:26,410::task::993::Storage.TaskManager.Task::(_decref) Task=`5db48345-288e-4814-9daf-90e20c978caa`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:52:26,411::__init__::232::IOProcess::(_processLogs) (7880) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:26,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:26,413::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:26,413::__init__::232::IOProcess::(_processLogs) (7881) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:26,414::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:26,414::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:26,415::__init__::232::IOProcess::(_processLogs) (7882) Queuing response >Dummy-33::DEBUG::2014-08-11 15:52:26,564::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:26,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134104 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:28,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:28,653::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132156 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:30,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:30,710::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133291 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:30,798::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:30,886::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:30,887::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:52:32,732::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:32,785::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.02806 s, 36.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:34,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:34,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133016 s, 77.0 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:52:35,848::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:52:35,849::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd69a347-115a-4128-859a-ad6da366c88f`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:35,885::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:35,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:35,896::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:35,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:52:35,898::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:35,899::task::1191::Storage.TaskManager.Task::(prepare) Task=`bd69a347-115a-4128-859a-ad6da366c88f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:35,899::task::595::Storage.TaskManager.Task::(_updateState) Task=`bd69a347-115a-4128-859a-ad6da366c88f`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:35,899::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:35,899::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:35,900::task::993::Storage.TaskManager.Task::(_decref) Task=`bd69a347-115a-4128-859a-ad6da366c88f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:52:35,910::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:52:35,911::task::595::Storage.TaskManager.Task::(_updateState) Task=`528f370e-23fd-45d7-b674-159ccaf11146`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:52:35,911::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:52:35,912::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`08e6ce5d-aca2-4b58-adde-76957b76c1a8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:52:35,912::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:52:35,912::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:52:35,913::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`08e6ce5d-aca2-4b58-adde-76957b76c1a8`::Granted request >Thread-34::DEBUG::2014-08-11 15:52:35,913::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`528f370e-23fd-45d7-b674-159ccaf11146`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:52:35,913::task::993::Storage.TaskManager.Task::(_decref) Task=`528f370e-23fd-45d7-b674-159ccaf11146`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:52:35,933::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:35,934::task::1191::Storage.TaskManager.Task::(prepare) Task=`528f370e-23fd-45d7-b674-159ccaf11146`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:35,934::task::595::Storage.TaskManager.Task::(_updateState) Task=`528f370e-23fd-45d7-b674-159ccaf11146`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:52:35,934::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:52:35,934::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:52:35,935::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:52:35,935::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:52:35,935::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:52:35,935::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:52:35,936::task::993::Storage.TaskManager.Task::(_decref) Task=`528f370e-23fd-45d7-b674-159ccaf11146`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:52:36,142::task::595::Storage.TaskManager.Task::(_updateState) Task=`396b5a1e-a40b-46a4-a369-a4d295acc01c`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:36,142::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:52:36,142::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000456691', 'lastCheck': '9.7', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:36,143::task::1191::Storage.TaskManager.Task::(prepare) Task=`396b5a1e-a40b-46a4-a369-a4d295acc01c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000456691', 'lastCheck': '9.7', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:36,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`396b5a1e-a40b-46a4-a369-a4d295acc01c`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:36,143::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:36,143::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:36,144::task::993::Storage.TaskManager.Task::(_decref) Task=`396b5a1e-a40b-46a4-a369-a4d295acc01c`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:52:36,166::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:52:36,417::__init__::232::IOProcess::(_processLogs) (7883) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:52:36,417::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:52:36,437::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000425807 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:36,438::__init__::232::IOProcess::(_processLogs) (7884) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:36,440::__init__::232::IOProcess::(_processLogs) (7885) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:36,441::__init__::232::IOProcess::(_processLogs) (7886) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:36,442::__init__::232::IOProcess::(_processLogs) (7887) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:52:37,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:37,046::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013268 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:39,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:39,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132464 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:40,903::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:40,904::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:40,986::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:52:41,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:41,158::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136105 s, 75.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:43,181::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:43,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134316 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:45,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:45,269::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133356 s, 76.8 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:52:45,948::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:52:45,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc4bb2b6-9c0d-4628-bbdd-627e36ee1155`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:45,986::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:45,993::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:45,995::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:45,996::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:52:45,999::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:45,999::task::1191::Storage.TaskManager.Task::(prepare) Task=`dc4bb2b6-9c0d-4628-bbdd-627e36ee1155`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:46,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`dc4bb2b6-9c0d-4628-bbdd-627e36ee1155`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:46,000::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:46,000::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:46,000::task::993::Storage.TaskManager.Task::(_decref) Task=`dc4bb2b6-9c0d-4628-bbdd-627e36ee1155`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:52:46,012::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:52:46,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e7d53f9-e9f3-47d9-a697-2006f364a451`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:52:46,013::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:52:46,014::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5aa590c9-b868-4cf3-8c16-c1141a28921f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:52:46,014::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:52:46,015::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:52:46,015::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5aa590c9-b868-4cf3-8c16-c1141a28921f`::Granted request >Thread-34::DEBUG::2014-08-11 15:52:46,016::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7e7d53f9-e9f3-47d9-a697-2006f364a451`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:52:46,016::task::993::Storage.TaskManager.Task::(_decref) Task=`7e7d53f9-e9f3-47d9-a697-2006f364a451`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:52:46,036::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:46,036::task::1191::Storage.TaskManager.Task::(prepare) Task=`7e7d53f9-e9f3-47d9-a697-2006f364a451`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:46,036::task::595::Storage.TaskManager.Task::(_updateState) Task=`7e7d53f9-e9f3-47d9-a697-2006f364a451`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:52:46,036::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:52:46,037::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:52:46,037::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:52:46,037::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:52:46,038::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:52:46,038::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:52:46,038::task::993::Storage.TaskManager.Task::(_decref) Task=`7e7d53f9-e9f3-47d9-a697-2006f364a451`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:52:46,444::__init__::232::IOProcess::(_processLogs) (7888) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:52:46,445::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:52:46,464::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000416915 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:46,466::__init__::232::IOProcess::(_processLogs) (7889) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:46,467::__init__::232::IOProcess::(_processLogs) (7890) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:46,468::__init__::232::IOProcess::(_processLogs) (7891) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:46,469::__init__::232::IOProcess::(_processLogs) (7892) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:52:47,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:47,326::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132256 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:49,348::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:49,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132319 s, 77.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:50,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:51,001::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:51,086::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:52:51,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`6dd0f11c-2a53-4dda-abe9-660890b3a27f`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:51,281::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:52:51,282::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416915', 'lastCheck': '4.8', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:51,282::task::1191::Storage.TaskManager.Task::(prepare) Task=`6dd0f11c-2a53-4dda-abe9-660890b3a27f`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000416915', 'lastCheck': '4.8', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:52:51,282::task::595::Storage.TaskManager.Task::(_updateState) Task=`6dd0f11c-2a53-4dda-abe9-660890b3a27f`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:51,283::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:51,283::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:51,283::task::993::Storage.TaskManager.Task::(_decref) Task=`6dd0f11c-2a53-4dda-abe9-660890b3a27f`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:52:51,304::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:52:51,405::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:51,438::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133763 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:53,461::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:53,495::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132753 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:55,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:55,552::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013411 s, 76.4 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:52:56,054::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:52:56,055::task::595::Storage.TaskManager.Task::(_updateState) Task=`aefc6b87-13c5-4009-9943-50913b7b7289`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:52:56,085::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:56,094::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:56,095::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:52:56,096::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:52:56,098::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:56,099::task::1191::Storage.TaskManager.Task::(prepare) Task=`aefc6b87-13c5-4009-9943-50913b7b7289`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:52:56,099::task::595::Storage.TaskManager.Task::(_updateState) Task=`aefc6b87-13c5-4009-9943-50913b7b7289`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:52:56,099::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:52:56,099::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:52:56,099::task::993::Storage.TaskManager.Task::(_decref) Task=`aefc6b87-13c5-4009-9943-50913b7b7289`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:52:56,113::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:52:56,114::task::595::Storage.TaskManager.Task::(_updateState) Task=`40439920-03eb-430c-ad5c-b565f1634fad`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:52:56,114::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:52:56,115::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f5b7961a-fe3b-40b2-b552-6b9ef6a7eaaf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:52:56,115::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:52:56,116::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:52:56,116::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f5b7961a-fe3b-40b2-b552-6b9ef6a7eaaf`::Granted request >Thread-34::DEBUG::2014-08-11 15:52:56,117::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`40439920-03eb-430c-ad5c-b565f1634fad`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:52:56,117::task::993::Storage.TaskManager.Task::(_decref) Task=`40439920-03eb-430c-ad5c-b565f1634fad`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:52:56,137::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:56,139::task::1191::Storage.TaskManager.Task::(prepare) Task=`40439920-03eb-430c-ad5c-b565f1634fad`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:52:56,140::task::595::Storage.TaskManager.Task::(_updateState) Task=`40439920-03eb-430c-ad5c-b565f1634fad`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:52:56,142::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:52:56,143::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:52:56,145::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:52:56,146::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:52:56,147::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:52:56,147::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:52:56,149::task::993::Storage.TaskManager.Task::(_decref) Task=`40439920-03eb-430c-ad5c-b565f1634fad`::ref 0 aborting False >Thread-16::DEBUG::2014-08-11 15:52:56,470::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:52:56,472::__init__::232::IOProcess::(_processLogs) (7893) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:52:56,480::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:52:56,481::__init__::232::IOProcess::(_processLogs) (7894) Got request for method 'touch' >Thread-16::DEBUG::2014-08-11 15:52:56,482::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:52:56,483::__init__::232::IOProcess::(_processLogs) (7895) Got request for method 'readfile' >Thread-16::DEBUG::2014-08-11 15:52:56,484::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:52:56,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:56,486::__init__::232::IOProcess::(_processLogs) (7896) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:56,486::__init__::232::IOProcess::(_processLogs) (7896) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:56,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:56,487::__init__::232::IOProcess::(_processLogs) (7897) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:52:56,488::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:52:56,488::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:52:56,489::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:52:56,490::__init__::232::IOProcess::(_processLogs) (7898) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:52:56,490::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:52:56,513::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00114192 s, 398 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:52:56,514::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:56,515::__init__::232::IOProcess::(_processLogs) (7899) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:52:56,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:56,517::__init__::232::IOProcess::(_processLogs) (7900) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:56,517::__init__::232::IOProcess::(_processLogs) (7900) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:56,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:52:56,518::__init__::232::IOProcess::(_processLogs) (7901) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:56,519::__init__::232::IOProcess::(_processLogs) (7901) Queuing response >Thread-14::DEBUG::2014-08-11 15:52:56,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:52:56,521::__init__::232::IOProcess::(_processLogs) (7902) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:52:56,522::__init__::232::IOProcess::(_processLogs) (7902) Queuing response >Dummy-33::DEBUG::2014-08-11 15:52:57,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:57,608::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131094 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:52:59,630::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:52:59,669::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0176906 s, 57.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:01,103::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:01,104::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:01,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:01,691::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:01,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013508 s, 75.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:03,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:03,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131484 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:05,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:05,839::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131882 s, 77.6 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:53:06,164::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:53:06,165::task::595::Storage.TaskManager.Task::(_updateState) Task=`79050eec-7f6d-4671-a7d9-b2029277b358`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:06,165::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-130::INFO::2014-08-11 15:53:06,178::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:06,178::task::1191::Storage.TaskManager.Task::(prepare) Task=`79050eec-7f6d-4671-a7d9-b2029277b358`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:06,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`79050eec-7f6d-4671-a7d9-b2029277b358`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:06,179::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:06,179::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:06,179::task::993::Storage.TaskManager.Task::(_decref) Task=`79050eec-7f6d-4671-a7d9-b2029277b358`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:53:06,190::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:53:06,191::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc0a8faf-9f8a-491a-90f4-0f46a17732d1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:53:06,191::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:53:06,192::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`af79dea5-9b4a-4dae-8d39-849fa0fefab2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:53:06,192::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:53:06,194::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:53:06,196::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`af79dea5-9b4a-4dae-8d39-849fa0fefab2`::Granted request >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:06,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:53:06,199::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bc0a8faf-9f8a-491a-90f4-0f46a17732d1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:53:06,200::task::993::Storage.TaskManager.Task::(_decref) Task=`bc0a8faf-9f8a-491a-90f4-0f46a17732d1`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:06,202::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:06,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:53:06,245::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:06,245::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc0a8faf-9f8a-491a-90f4-0f46a17732d1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:06,245::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc0a8faf-9f8a-491a-90f4-0f46a17732d1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:53:06,246::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:53:06,246::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:53:06,246::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:53:06,247::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:53:06,247::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:53:06,247::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:53:06,247::task::993::Storage.TaskManager.Task::(_decref) Task=`bc0a8faf-9f8a-491a-90f4-0f46a17732d1`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:53:06,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`1678ca09-ee27-44fd-9e8e-2e57db7eacb9`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:06,397::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:53:06,397::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00114192', 'lastCheck': '9.9', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:06,398::task::1191::Storage.TaskManager.Task::(prepare) Task=`1678ca09-ee27-44fd-9e8e-2e57db7eacb9`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00114192', 'lastCheck': '9.9', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:06,398::task::595::Storage.TaskManager.Task::(_updateState) Task=`1678ca09-ee27-44fd-9e8e-2e57db7eacb9`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:06,398::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:06,398::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:06,399::task::993::Storage.TaskManager.Task::(_decref) Task=`1678ca09-ee27-44fd-9e8e-2e57db7eacb9`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:53:06,418::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:53:06,523::__init__::232::IOProcess::(_processLogs) (7903) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:53:06,524::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:53:06,544::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000437255 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:53:06,545::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:06,545::__init__::232::IOProcess::(_processLogs) (7904) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:53:06,547::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:06,547::__init__::232::IOProcess::(_processLogs) (7905) Queuing response >Thread-14::DEBUG::2014-08-11 15:53:06,548::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:06,548::__init__::232::IOProcess::(_processLogs) (7906) Queuing response >Thread-14::DEBUG::2014-08-11 15:53:06,549::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:06,549::__init__::232::IOProcess::(_processLogs) (7907) Queuing response >Dummy-33::DEBUG::2014-08-11 15:53:07,863::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:07,896::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132447 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:09,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:09,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132574 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:11,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:11,226::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:11,286::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:11,974::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:12,008::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131322 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:14,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:14,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132592 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:16,086::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:16,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013253 s, 77.3 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:53:16,261::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:53:16,262::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3163c7d-65bc-47d5-8af3-9d1b934b90c7`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:16,285::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:16,293::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:16,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:16,296::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:53:16,298::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:16,299::task::1191::Storage.TaskManager.Task::(prepare) Task=`c3163c7d-65bc-47d5-8af3-9d1b934b90c7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:16,299::task::595::Storage.TaskManager.Task::(_updateState) Task=`c3163c7d-65bc-47d5-8af3-9d1b934b90c7`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:16,299::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:16,299::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:16,299::task::993::Storage.TaskManager.Task::(_decref) Task=`c3163c7d-65bc-47d5-8af3-9d1b934b90c7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:53:16,314::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:53:16,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`63de20d8-8cb6-47fe-9cc1-249cb725341c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:53:16,315::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:53:16,316::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3976cbd1-c251-46f9-b293-f0e2fcae7fb3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:53:16,316::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:53:16,319::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:53:16,319::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`3976cbd1-c251-46f9-b293-f0e2fcae7fb3`::Granted request >Thread-34::DEBUG::2014-08-11 15:53:16,320::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`63de20d8-8cb6-47fe-9cc1-249cb725341c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:53:16,320::task::993::Storage.TaskManager.Task::(_decref) Task=`63de20d8-8cb6-47fe-9cc1-249cb725341c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:53:16,340::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:16,340::task::1191::Storage.TaskManager.Task::(prepare) Task=`63de20d8-8cb6-47fe-9cc1-249cb725341c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:16,340::task::595::Storage.TaskManager.Task::(_updateState) Task=`63de20d8-8cb6-47fe-9cc1-249cb725341c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:53:16,340::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:53:16,341::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:53:16,341::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:53:16,341::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:53:16,342::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:53:16,342::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:53:16,342::task::993::Storage.TaskManager.Task::(_decref) Task=`63de20d8-8cb6-47fe-9cc1-249cb725341c`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:53:16,551::__init__::232::IOProcess::(_processLogs) (7908) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:53:16,552::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:53:16,572::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000434074 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:53:16,573::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:16,573::__init__::232::IOProcess::(_processLogs) (7909) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:53:16,574::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:16,575::__init__::232::IOProcess::(_processLogs) (7910) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:16,575::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:16,576::__init__::232::IOProcess::(_processLogs) (7911) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:16,576::__init__::232::IOProcess::(_processLogs) (7911) Queuing response >Thread-14::DEBUG::2014-08-11 15:53:16,577::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:16,577::__init__::232::IOProcess::(_processLogs) (7912) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:16,578::__init__::232::IOProcess::(_processLogs) (7912) Queuing response >Dummy-33::DEBUG::2014-08-11 15:53:18,141::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:18,175::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132997 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:20,197::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:20,231::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133113 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:21,300::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:21,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:21,386::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:53:21,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ab34d4c-9d9b-4b6f-ba47-8964899b0fc2`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:21,515::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:53:21,516::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000434074', 'lastCheck': '4.9', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:21,516::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ab34d4c-9d9b-4b6f-ba47-8964899b0fc2`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000434074', 'lastCheck': '4.9', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:21,516::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ab34d4c-9d9b-4b6f-ba47-8964899b0fc2`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:21,516::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:21,516::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:21,517::task::993::Storage.TaskManager.Task::(_decref) Task=`5ab34d4c-9d9b-4b6f-ba47-8964899b0fc2`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:53:21,537::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:22,254::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:22,288::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131681 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:24,328::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:24,361::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132893 s, 77.1 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:53:26,356::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:53:26,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`76b081c0-8e72-44c0-a1a1-5352f821fcd0`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:26,357::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-130::INFO::2014-08-11 15:53:26,370::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:26,370::task::1191::Storage.TaskManager.Task::(prepare) Task=`76b081c0-8e72-44c0-a1a1-5352f821fcd0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:26,370::task::595::Storage.TaskManager.Task::(_updateState) Task=`76b081c0-8e72-44c0-a1a1-5352f821fcd0`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:26,371::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:26,371::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:26,371::task::993::Storage.TaskManager.Task::(_decref) Task=`76b081c0-8e72-44c0-a1a1-5352f821fcd0`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:53:26,381::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:53:26,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`6df0fa0a-fb96-4d15-8c1f-bfdf58771591`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:53:26,382::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:53:26,383::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`876ca77f-1dd9-4978-a958-11230121b4ed`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:53:26,383::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:53:26,384::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:53:26,384::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`876ca77f-1dd9-4978-a958-11230121b4ed`::Granted request >Dummy-33::DEBUG::2014-08-11 15:53:26,385::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-34::DEBUG::2014-08-11 15:53:26,386::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6df0fa0a-fb96-4d15-8c1f-bfdf58771591`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:53:26,406::task::993::Storage.TaskManager.Task::(_decref) Task=`6df0fa0a-fb96-4d15-8c1f-bfdf58771591`::ref 1 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:26,409::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:26,414::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:26,419::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:26,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0344964 s, 29.7 MB/s\n'; <rc> = 0 >Thread-34::INFO::2014-08-11 15:53:26,458::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:26,459::task::1191::Storage.TaskManager.Task::(prepare) Task=`6df0fa0a-fb96-4d15-8c1f-bfdf58771591`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951887872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:26,460::task::595::Storage.TaskManager.Task::(_updateState) Task=`6df0fa0a-fb96-4d15-8c1f-bfdf58771591`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:53:26,461::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:53:26,462::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:53:26,464::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:53:26,466::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:53:26,467::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:53:26,467::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:53:26,469::task::993::Storage.TaskManager.Task::(_decref) Task=`6df0fa0a-fb96-4d15-8c1f-bfdf58771591`::ref 0 aborting False >Thread-122::DEBUG::2014-08-11 15:53:26,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6d54f0d-9378-42aa-8370-7a3006766a45`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:53:26,573::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:53:26,574::__init__::232::IOProcess::(_processLogs) (7913) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:53:26,576::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:26,576::__init__::232::IOProcess::(_processLogs) (7914) Got request for method 'stat' >Thread-122::INFO::2014-08-11 15:53:26,577::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:53:26,577::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6d54f0d-9378-42aa-8370-7a3006766a45`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:53:26,577::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6d54f0d-9378-42aa-8370-7a3006766a45`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:53:26,578::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:53:26,578::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:53:26,578::task::993::Storage.TaskManager.Task::(_decref) Task=`b6d54f0d-9378-42aa-8370-7a3006766a45`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:53:26,579::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:53:26,580::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:26,580::__init__::232::IOProcess::(_processLogs) (7915) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:53:26,581::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:53:26,602::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000783746 s, 581 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:53:26,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:53:26,604::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:26,604::__init__::232::IOProcess::(_processLogs) (7916) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:53:26,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:53:26,606::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:26,607::__init__::232::IOProcess::(_processLogs) (7917) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:26,607::__init__::232::IOProcess::(_processLogs) (7917) Queuing response >Thread-14::DEBUG::2014-08-11 15:53:26,608::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:53:26,608::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:26,608::__init__::232::IOProcess::(_processLogs) (7918) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:26,609::__init__::232::IOProcess::(_processLogs) (7918) Queuing response >Thread-14::DEBUG::2014-08-11 15:53:26,609::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:53:26,610::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:26,610::__init__::232::IOProcess::(_processLogs) (7919) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:26,610::__init__::232::IOProcess::(_processLogs) (7919) Queuing response >Dummy-33::DEBUG::2014-08-11 15:53:28,484::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:28,518::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133742 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:30,540::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:30,574::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134125 s, 76.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:31,430::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:31,432::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:31,433::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:32,598::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:32,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0185013 s, 55.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:34,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:34,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131335 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:36,440::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:36,441::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:53:36,484::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:53:36,488::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae41fe1e-3829-4752-bea6-dfda8ccd3656`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:36,489::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:53:36,491::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-130::INFO::2014-08-11 15:53:36,505::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:36,506::task::1191::Storage.TaskManager.Task::(prepare) Task=`ae41fe1e-3829-4752-bea6-dfda8ccd3656`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:36,506::task::595::Storage.TaskManager.Task::(_updateState) Task=`ae41fe1e-3829-4752-bea6-dfda8ccd3656`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:36,506::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:36,506::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:36,506::task::993::Storage.TaskManager.Task::(_decref) Task=`ae41fe1e-3829-4752-bea6-dfda8ccd3656`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:53:36,518::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:53:36,519::task::595::Storage.TaskManager.Task::(_updateState) Task=`383ec2c7-fc81-4b33-a059-ced72f5cd01c`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:53:36,519::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:53:36,520::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f3bb0a76-d8c6-4770-ac3d-788e6a423b4a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:53:36,520::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:53:36,521::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:53:36,521::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f3bb0a76-d8c6-4770-ac3d-788e6a423b4a`::Granted request >Thread-34::DEBUG::2014-08-11 15:53:36,521::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`383ec2c7-fc81-4b33-a059-ced72f5cd01c`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:53:36,522::task::993::Storage.TaskManager.Task::(_decref) Task=`383ec2c7-fc81-4b33-a059-ced72f5cd01c`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:53:36,541::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951822336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:36,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`383ec2c7-fc81-4b33-a059-ced72f5cd01c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852951822336', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:36,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`383ec2c7-fc81-4b33-a059-ced72f5cd01c`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:53:36,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:53:36,542::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:53:36,543::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:53:36,543::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:53:36,543::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:53:36,543::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:53:36,544::task::993::Storage.TaskManager.Task::(_decref) Task=`383ec2c7-fc81-4b33-a059-ced72f5cd01c`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:53:36,612::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:36,612::__init__::232::IOProcess::(_processLogs) (7920) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:53:36,613::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:53:36,633::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000418805 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:53:36,634::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:36,634::__init__::232::IOProcess::(_processLogs) (7921) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:53:36,635::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:36,636::__init__::232::IOProcess::(_processLogs) (7922) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:36,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:36,637::__init__::232::IOProcess::(_processLogs) (7923) Queuing response >Thread-14::DEBUG::2014-08-11 15:53:36,638::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:36,638::__init__::232::IOProcess::(_processLogs) (7924) Got request for method 'access' >Thread-130::DEBUG::2014-08-11 15:53:36,657::task::595::Storage.TaskManager.Task::(_updateState) Task=`dac47f60-6460-48d0-b997-90962ab0296b`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:36,657::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:53:36,657::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418805', 'lastCheck': '0.0', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:36,658::task::1191::Storage.TaskManager.Task::(prepare) Task=`dac47f60-6460-48d0-b997-90962ab0296b`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000418805', 'lastCheck': '0.0', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:36,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`dac47f60-6460-48d0-b997-90962ab0296b`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:36,658::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:36,658::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:36,658::task::993::Storage.TaskManager.Task::(_decref) Task=`dac47f60-6460-48d0-b997-90962ab0296b`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:53:36,679::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:36,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:36,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133784 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:38,955::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:38,989::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013294 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:41,012::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:41,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132923 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:41,499::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:41,500::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:41,586::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:43,068::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:43,102::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131847 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:45,125::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:45,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0215816 s, 47.4 MB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:53:46,556::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:53:46,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cb826d4-284a-4a67-b3b4-5f1359ee7396`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:46,585::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:46,594::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:46,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:46,596::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::INFO::2014-08-11 15:53:46,599::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:46,599::task::1191::Storage.TaskManager.Task::(prepare) Task=`6cb826d4-284a-4a67-b3b4-5f1359ee7396`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:46,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cb826d4-284a-4a67-b3b4-5f1359ee7396`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:46,600::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:46,600::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:46,600::task::993::Storage.TaskManager.Task::(_decref) Task=`6cb826d4-284a-4a67-b3b4-5f1359ee7396`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:53:46,614::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:53:46,615::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d332c53-754b-41af-aef6-73d4822cd508`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:53:46,615::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:53:46,616::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ee257405-de62-4568-8fd5-a0afeb291a0e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:53:46,616::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:53:46,617::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:53:46,617::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ee257405-de62-4568-8fd5-a0afeb291a0e`::Granted request >Thread-34::DEBUG::2014-08-11 15:53:46,617::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1d332c53-754b-41af-aef6-73d4822cd508`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:53:46,618::task::993::Storage.TaskManager.Task::(_decref) Task=`1d332c53-754b-41af-aef6-73d4822cd508`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:53:46,638::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852948545536', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:46,638::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d332c53-754b-41af-aef6-73d4822cd508`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852948545536', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:46,638::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d332c53-754b-41af-aef6-73d4822cd508`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:53:46,638::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:53:46,639::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:53:46,639::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:53:46,639::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:53:46,640::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:53:46,641::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-14::DEBUG::2014-08-11 15:53:46,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:53:46,644::__init__::232::IOProcess::(_processLogs) (7925) Got request for method 'statvfs' >Thread-34::DEBUG::2014-08-11 15:53:46,644::task::993::Storage.TaskManager.Task::(_decref) Task=`1d332c53-754b-41af-aef6-73d4822cd508`::ref 0 aborting False >Thread-16::DEBUG::2014-08-11 15:53:46,645::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:53:46,645::__init__::232::IOProcess::(_processLogs) (7925) Queuing response >Thread-16::DEBUG::2014-08-11 15:53:46,669::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.0011765 s, 387 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:53:46,670::__init__::232::IOProcess::(_processLogs) (7926) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:53:46,672::__init__::232::IOProcess::(_processLogs) (7927) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:46,673::__init__::232::IOProcess::(_processLogs) (7928) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:46,674::__init__::232::IOProcess::(_processLogs) (7929) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:53:47,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:47,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131962 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:49,246::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:49,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132514 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:51,303::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:51,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131975 s, 77.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:51,602::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:51,604::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:51,686::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:53:51,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`79cc362f-43a5-4fb0-8622-cfef21f085f8`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:53:51,777::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:53:51,777::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0011765', 'lastCheck': '5.1', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:51,777::task::1191::Storage.TaskManager.Task::(prepare) Task=`79cc362f-43a5-4fb0-8622-cfef21f085f8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.0011765', 'lastCheck': '5.1', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:53:51,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`79cc362f-43a5-4fb0-8622-cfef21f085f8`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:53:51,778::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:51,778::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:53:51,778::task::993::Storage.TaskManager.Task::(_decref) Task=`79cc362f-43a5-4fb0-8622-cfef21f085f8`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:53:51,800::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:53:53,359::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:53,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0278652 s, 36.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:55,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:55,629::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135053 s, 75.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:53:56,676::__init__::232::IOProcess::(_processLogs) (7930) Got request for method 'statvfs' >Thread-130::DEBUG::2014-08-11 15:53:56,685::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:53:56,689::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce230845-3528-4f80-a23a-5b1d09a6cb62`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 15:53:56,689::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-130::INFO::2014-08-11 15:53:56,690::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:56,713::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:56,715::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:53:56,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000854249 s, 533 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:53:56,719::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:53:56,721::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:56,722::__init__::232::IOProcess::(_processLogs) (7931) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:53:56,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:53:56,726::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:56,727::__init__::232::IOProcess::(_processLogs) (7932) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:53:56,727::__init__::232::IOProcess::(_processLogs) (7932) Queuing response >Thread-14::DEBUG::2014-08-11 15:53:56,729::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:53:56,729::__init__::232::IOProcess::(_processLogs) (7933) Got request for method 'access' >Thread-130::INFO::2014-08-11 15:53:56,730::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:53:56,732::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce230845-3528-4f80-a23a-5b1d09a6cb62`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:53:56,732::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-130::DEBUG::2014-08-11 15:53:56,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce230845-3528-4f80-a23a-5b1d09a6cb62`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:53:56,734::__init__::232::IOProcess::(_processLogs) (7934) Got request for method 'access' >Thread-130::DEBUG::2014-08-11 15:53:56,734::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:53:56,735::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 15:53:56,735::__init__::232::IOProcess::(_processLogs) (7934) Queuing response >Thread-130::DEBUG::2014-08-11 15:53:56,736::task::993::Storage.TaskManager.Task::(_decref) Task=`ce230845-3528-4f80-a23a-5b1d09a6cb62`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:53:56,749::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:53:56,750::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8def25f-7e13-4071-bdb0-2db39d2fde96`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:53:56,750::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:53:56,751::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`912a0ab8-b0a6-46c9-885e-87fa39703484`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:53:56,751::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:53:56,752::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:53:56,752::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`912a0ab8-b0a6-46c9-885e-87fa39703484`::Granted request >Thread-34::DEBUG::2014-08-11 15:53:56,753::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e8def25f-7e13-4071-bdb0-2db39d2fde96`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:53:56,753::task::993::Storage.TaskManager.Task::(_decref) Task=`e8def25f-7e13-4071-bdb0-2db39d2fde96`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:53:56,773::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946251776', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:56,773::task::1191::Storage.TaskManager.Task::(prepare) Task=`e8def25f-7e13-4071-bdb0-2db39d2fde96`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946251776', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:53:56,774::task::595::Storage.TaskManager.Task::(_updateState) Task=`e8def25f-7e13-4071-bdb0-2db39d2fde96`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:53:56,774::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:53:56,774::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:53:56,775::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:53:56,775::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:53:56,775::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:53:56,775::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:53:56,776::task::993::Storage.TaskManager.Task::(_decref) Task=`e8def25f-7e13-4071-bdb0-2db39d2fde96`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:53:57,652::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:57,686::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131043 s, 78.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:53:59,708::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:53:59,741::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133627 s, 76.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:01,732::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:01,733::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:01,765::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:01,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:01,820::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136069 s, 75.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:03,843::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:03,877::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131859 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:05,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:05,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131568 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:54:06,741::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:06,742::__init__::232::IOProcess::(_processLogs) (7935) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:06,786::__init__::232::IOProcess::(_processLogs) (7935) Queuing response >Thread-16::DEBUG::2014-08-11 15:54:06,787::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-130::DEBUG::2014-08-11 15:54:06,806::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:54:06,808::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ed915bd-0636-4ffc-8e00-7762224e8c08`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 15:54:06,809::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000536028 s, 849 kB/s\n'; <rc> = 0 >Thread-130::INFO::2014-08-11 15:54:06,810::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:54:06,812::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:06,814::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:06,813::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:06,816::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:06,817::__init__::232::IOProcess::(_processLogs) (7936) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:06,822::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:06,824::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:06,826::__init__::232::IOProcess::(_processLogs) (7937) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:06,827::__init__::232::IOProcess::(_processLogs) (7937) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:06,828::__init__::232::IOProcess::(_processLogs) (7938) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:06,830::__init__::232::IOProcess::(_processLogs) (7939) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:06,830::__init__::232::IOProcess::(_processLogs) (7939) Queuing response >Thread-130::INFO::2014-08-11 15:54:06,830::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:54:06,832::task::1191::Storage.TaskManager.Task::(prepare) Task=`9ed915bd-0636-4ffc-8e00-7762224e8c08`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:54:06,833::task::595::Storage.TaskManager.Task::(_updateState) Task=`9ed915bd-0636-4ffc-8e00-7762224e8c08`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:54:06,833::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:54:06,833::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:54:06,834::task::993::Storage.TaskManager.Task::(_decref) Task=`9ed915bd-0636-4ffc-8e00-7762224e8c08`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:54:06,844::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:54:06,845::task::595::Storage.TaskManager.Task::(_updateState) Task=`9646206d-f073-4e7b-8987-e57a63666ca2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:54:06,845::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:54:06,846::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`26728d9b-d1b5-419c-a0ca-ec387d6ff188`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:54:06,846::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:54:06,847::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:54:06,847::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`26728d9b-d1b5-419c-a0ca-ec387d6ff188`::Granted request >Thread-34::DEBUG::2014-08-11 15:54:06,848::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9646206d-f073-4e7b-8987-e57a63666ca2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:54:06,848::task::993::Storage.TaskManager.Task::(_decref) Task=`9646206d-f073-4e7b-8987-e57a63666ca2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:54:06,868::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946186240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:06,868::task::1191::Storage.TaskManager.Task::(prepare) Task=`9646206d-f073-4e7b-8987-e57a63666ca2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946186240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:06,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`9646206d-f073-4e7b-8987-e57a63666ca2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:54:06,868::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:54:06,869::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:54:06,869::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:54:06,869::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:54:06,870::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:54:06,870::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:54:06,870::task::993::Storage.TaskManager.Task::(_decref) Task=`9646206d-f073-4e7b-8987-e57a63666ca2`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:54:06,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`0633a4d2-a669-480a-83ff-2cc8ad39046e`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:54:06,903::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:54:06,903::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000536028', 'lastCheck': '0.1', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:54:06,904::task::1191::Storage.TaskManager.Task::(prepare) Task=`0633a4d2-a669-480a-83ff-2cc8ad39046e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000536028', 'lastCheck': '0.1', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:54:06,904::task::595::Storage.TaskManager.Task::(_updateState) Task=`0633a4d2-a669-480a-83ff-2cc8ad39046e`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:54:06,904::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:54:06,904::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:54:06,904::task::993::Storage.TaskManager.Task::(_decref) Task=`0633a4d2-a669-480a-83ff-2cc8ad39046e`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:54:06,924::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:07,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:07,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133068 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:10,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:10,050::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133382 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:11,841::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:11,844::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:11,845::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:12,073::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:12,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132686 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:14,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:14,163::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132465 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:16,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:16,219::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131847 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:54:16,834::__init__::232::IOProcess::(_processLogs) (7940) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:16,852::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:16,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:16,890::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:54:16,884::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-130::DEBUG::2014-08-11 15:54:16,893::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3a5ae00-79d1-4166-bf17-b51f4df9d522`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:54:16,894::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-16::DEBUG::2014-08-11 15:54:16,895::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:54:16,935::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000404882 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:54:16,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:16,940::__init__::232::IOProcess::(_processLogs) (7941) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:16,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:16,942::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:16,942::__init__::232::IOProcess::(_processLogs) (7942) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:16,943::__init__::232::IOProcess::(_processLogs) DEBUG|(7942) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:16,945::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:16,945::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:16,946::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:16,947::__init__::232::IOProcess::(_processLogs) (7943) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:16,947::__init__::232::IOProcess::(_processLogs) (7943) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:16,948::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:16,948::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:16,948::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:16,949::__init__::232::IOProcess::(_processLogs) (7944) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:16,949::__init__::232::IOProcess::(_processLogs) (7944) Queuing response >Thread-130::INFO::2014-08-11 15:54:16,957::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:54:16,958::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3a5ae00-79d1-4166-bf17-b51f4df9d522`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:54:16,958::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3a5ae00-79d1-4166-bf17-b51f4df9d522`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:54:16,958::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:54:16,958::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:54:16,959::task::993::Storage.TaskManager.Task::(_decref) Task=`d3a5ae00-79d1-4166-bf17-b51f4df9d522`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:54:16,970::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:54:16,970::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bf02a43-7c9c-467c-a0cb-1778f0cf7efa`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:54:16,971::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:54:16,971::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`10492d50-cce4-4412-a6ac-9d2180b26064`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:54:16,972::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:54:16,972::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:54:16,972::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`10492d50-cce4-4412-a6ac-9d2180b26064`::Granted request >Thread-34::DEBUG::2014-08-11 15:54:16,973::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`8bf02a43-7c9c-467c-a0cb-1778f0cf7efa`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:54:16,973::task::993::Storage.TaskManager.Task::(_decref) Task=`8bf02a43-7c9c-467c-a0cb-1778f0cf7efa`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:54:16,993::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946186240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:16,993::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bf02a43-7c9c-467c-a0cb-1778f0cf7efa`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946186240', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:16,993::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bf02a43-7c9c-467c-a0cb-1778f0cf7efa`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:54:16,994::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:54:16,994::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:54:16,994::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:54:16,995::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:54:16,995::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:54:16,995::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:54:16,995::task::993::Storage.TaskManager.Task::(_decref) Task=`8bf02a43-7c9c-467c-a0cb-1778f0cf7efa`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:54:18,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:18,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132653 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:20,298::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:20,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133393 s, 76.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:21,898::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:21,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:21,990::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:54:22,019::task::595::Storage.TaskManager.Task::(_updateState) Task=`60c90d53-07cd-4a36-b572-7a7cb1c059bf`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:54:22,020::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-130::INFO::2014-08-11 15:54:22,021::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000404882', 'lastCheck': '5.1', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:54:22,022::task::1191::Storage.TaskManager.Task::(prepare) Task=`60c90d53-07cd-4a36-b572-7a7cb1c059bf`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000404882', 'lastCheck': '5.1', 'valid': True}} >Thread-130::DEBUG::2014-08-11 15:54:22,022::task::595::Storage.TaskManager.Task::(_updateState) Task=`60c90d53-07cd-4a36-b572-7a7cb1c059bf`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:54:22,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:54:22,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:54:22,024::task::993::Storage.TaskManager.Task::(_decref) Task=`60c90d53-07cd-4a36-b572-7a7cb1c059bf`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:54:22,048::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:22,354::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:22,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131379 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:24,410::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:24,443::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131244 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:26,466::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:26,500::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133159 s, 76.9 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:54:26,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`7672bc31-c348-4376-a1ec-6f8b87c52bf1`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:54:26,823::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:54:26,824::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:26,824::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:26,825::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:26,825::__init__::232::IOProcess::(_processLogs) (7945) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:54:26,825::__init__::232::IOProcess::(_processLogs) DEBUG|(7945) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:26,826::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:26,827::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:26,827::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:26,827::__init__::232::IOProcess::(_processLogs) DEBUG|(7946) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:54:26,828::__init__::232::IOProcess::(_processLogs) (7946) Queuing response >Thread-122::INFO::2014-08-11 15:54:26,828::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:54:26,828::task::1191::Storage.TaskManager.Task::(prepare) Task=`7672bc31-c348-4376-a1ec-6f8b87c52bf1`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:54:26,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`7672bc31-c348-4376-a1ec-6f8b87c52bf1`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:54:26,829::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:54:26,829::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:54:26,829::task::993::Storage.TaskManager.Task::(_decref) Task=`7672bc31-c348-4376-a1ec-6f8b87c52bf1`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:54:26,958::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:26,958::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:26,985::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:26,986::__init__::232::IOProcess::(_processLogs) (7947) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:26,986::__init__::232::IOProcess::(_processLogs) (7947) Queuing response >Thread-16::DEBUG::2014-08-11 15:54:26,987::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:27,007::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-130::DEBUG::2014-08-11 15:54:27,011::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-16::DEBUG::2014-08-11 15:54:27,011::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000574 s, 793 kB/s\n'; <rc> = 0 >Thread-130::DEBUG::2014-08-11 15:54:27,014::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5135500-9a80-425f-950e-ac2b2ec48c59`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:54:27,016::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:54:27,017::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:27,018::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:27,019::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:27,020::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:27,020::__init__::232::IOProcess::(_processLogs) (7948) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:27,023::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:27,025::__init__::232::IOProcess::(_processLogs) (7948) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:27,028::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:27,028::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:27,029::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:27,029::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(7949) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:27,029::__init__::232::IOProcess::(_processLogs) (7949) Queuing response >Thread-130::INFO::2014-08-11 15:54:27,030::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:54:27,031::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-130::DEBUG::2014-08-11 15:54:27,032::task::1191::Storage.TaskManager.Task::(prepare) Task=`b5135500-9a80-425f-950e-ac2b2ec48c59`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:54:27,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-130::DEBUG::2014-08-11 15:54:27,033::task::595::Storage.TaskManager.Task::(_updateState) Task=`b5135500-9a80-425f-950e-ac2b2ec48c59`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:54:27,034::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-130::DEBUG::2014-08-11 15:54:27,034::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:54:27,035::__init__::232::IOProcess::(_processLogs) (7950) Got request for method 'access' >Thread-130::DEBUG::2014-08-11 15:54:27,035::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 15:54:27,037::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|(7950) Queuing response >Thread-130::DEBUG::2014-08-11 15:54:27,037::task::993::Storage.TaskManager.Task::(_decref) Task=`b5135500-9a80-425f-950e-ac2b2ec48c59`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:54:27,039::__init__::232::IOProcess::(_processLogs) DEBUGDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:27,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:27,041::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:27,041::__init__::232::IOProcess::(_processLogs) (7951) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:27,042::__init__::232::IOProcess::(_processLogs) (7951) Queuing response >Thread-34::DEBUG::2014-08-11 15:54:27,063::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:54:27,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe3d25ae-85b6-48ad-a398-73fe1c77e427`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:54:27,068::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:54:27,070::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`836fbf4d-9ed1-4467-9e14-a9c54eb7dda6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:54:27,070::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:54:27,071::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:54:27,071::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`836fbf4d-9ed1-4467-9e14-a9c54eb7dda6`::Granted request >Thread-34::DEBUG::2014-08-11 15:54:27,072::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fe3d25ae-85b6-48ad-a398-73fe1c77e427`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:54:27,073::task::993::Storage.TaskManager.Task::(_decref) Task=`fe3d25ae-85b6-48ad-a398-73fe1c77e427`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:54:27,110::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946120704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:27,111::task::1191::Storage.TaskManager.Task::(prepare) Task=`fe3d25ae-85b6-48ad-a398-73fe1c77e427`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946120704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:27,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`fe3d25ae-85b6-48ad-a398-73fe1c77e427`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:54:27,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:54:27,112::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:54:27,113::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:54:27,114::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:54:27,114::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:54:27,114::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:54:27,115::task::993::Storage.TaskManager.Task::(_decref) Task=`fe3d25ae-85b6-48ad-a398-73fe1c77e427`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:54:28,523::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:28,557::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134469 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:30,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:30,613::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133007 s, 77.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:32,035::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:32,041::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:32,089::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:32,654::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:32,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0276845 s, 37.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:34,895::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:34,929::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132863 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:36,952::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:36,986::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132491 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:54:37,043::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:37,044::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:37,086::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Extracting request information... >Thread-16::DEBUG::2014-08-11 15:54:37,087::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:54:37,088::__init__::232::IOProcess::(_processLogs) (7952) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:37,107::__init__::232::IOProcess::(_processLogs) (7952) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:37,111::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:54:37,111::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000505526 s, 900 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:54:37,114::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:37,115::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:37,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:37,119::__init__::232::IOProcess::(_processLogs) (7953) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:37,119::__init__::232::IOProcess::(_processLogs) (7953) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:37,119::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:37,120::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:37,121::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:37,122::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:37,123::__init__::232::IOProcess::(_processLogs) (7954) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:37,125::__init__::232::IOProcess::(_processLogs) (7954) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:37,126::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:37,128::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-130::DEBUG::2014-08-11 15:54:37,130::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 15:54:37,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:37,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-130::DEBUG::2014-08-11 15:54:37,133::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd89b57d-c97e-433c-988a-a3640c451507`::moving from state init -> state preparing >Thread-130::INFO::2014-08-11 15:54:37,135::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:54:37,136::__init__::232::IOProcess::(_processLogs) (7955) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:37,137::__init__::232::IOProcess::(_processLogs) (7955) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:37,137::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:37,137::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:37,139::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:37,139::__init__::232::IOProcess::(_processLogs) (7956) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:37,140::__init__::232::IOProcess::(_processLogs) (7956) Queuing response >Detector thread::DEBUG::2014-08-11 15:54:37,140::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:38571 >Detector thread::DEBUG::2014-08-11 15:54:37,141::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:38571 >Detector thread::DEBUG::2014-08-11 15:54:37,142::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:38571 >Detector thread::DEBUG::2014-08-11 15:54:37,143::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 38571) >Thread-132::DEBUG::2014-08-11 15:54:37,147::task::595::Storage.TaskManager.Task::(_updateState) Task=`005ed978-91f1-4946-a606-9271d79f0109`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:54:37,148::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-08-11 15:54:37,148::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000505526', 'lastCheck': '0.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:54:37,148::task::1191::Storage.TaskManager.Task::(prepare) Task=`005ed978-91f1-4946-a606-9271d79f0109`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000505526', 'lastCheck': '0.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:54:37,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`005ed978-91f1-4946-a606-9271d79f0109`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:54:37,149::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:54:37,149::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::INFO::2014-08-11 15:54:37,150::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:54:37,150::task::993::Storage.TaskManager.Task::(_decref) Task=`005ed978-91f1-4946-a606-9271d79f0109`::ref 0 aborting False >Thread-130::DEBUG::2014-08-11 15:54:37,151::task::1191::Storage.TaskManager.Task::(prepare) Task=`dd89b57d-c97e-433c-988a-a3640c451507`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-130::DEBUG::2014-08-11 15:54:37,152::task::595::Storage.TaskManager.Task::(_updateState) Task=`dd89b57d-c97e-433c-988a-a3640c451507`::moving from state preparing -> state finished >Thread-130::DEBUG::2014-08-11 15:54:37,153::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-130::DEBUG::2014-08-11 15:54:37,155::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-130::DEBUG::2014-08-11 15:54:37,156::task::993::Storage.TaskManager.Task::(_decref) Task=`dd89b57d-c97e-433c-988a-a3640c451507`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:54:37,168::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:54:37,168::task::595::Storage.TaskManager.Task::(_updateState) Task=`04c3da56-de85-4c88-9c10-0cde69dfecb7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:54:37,169::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:54:37,170::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c93083e0-4853-4e10-bf43-73288d1dc189`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:54:37,170::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:54:37,170::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:54:37,171::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c93083e0-4853-4e10-bf43-73288d1dc189`::Granted request >Thread-34::DEBUG::2014-08-11 15:54:37,171::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`04c3da56-de85-4c88-9c10-0cde69dfecb7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:54:37,171::task::993::Storage.TaskManager.Task::(_decref) Task=`04c3da56-de85-4c88-9c10-0cde69dfecb7`::ref 1 aborting False >Thread-132::DEBUG::2014-08-11 15:54:37,183::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:54:37,198::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946120704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:37,198::task::1191::Storage.TaskManager.Task::(prepare) Task=`04c3da56-de85-4c88-9c10-0cde69dfecb7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852946120704', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:37,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`04c3da56-de85-4c88-9c10-0cde69dfecb7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:54:37,199::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:54:37,199::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:54:37,199::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:54:37,200::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:54:37,200::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:54:37,200::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:54:37,201::task::993::Storage.TaskManager.Task::(_decref) Task=`04c3da56-de85-4c88-9c10-0cde69dfecb7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:54:39,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:39,042::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131858 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:41,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:41,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131398 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:42,141::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:42,143::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:42,186::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:43,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:43,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132076 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:45,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:45,210::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131836 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:54:47,141::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:47,142::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:54:47,143::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:54:47,144::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:47,209::__init__::232::IOProcess::(_processLogs) (7957) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:47,214::__init__::232::IOProcess::(_processLogs) (7957) Queuing response >Thread-16::DEBUG::2014-08-11 15:54:47,216::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000862986 s, 527 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:47,216::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:54:47,220::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 15:54:47,223::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-132::DEBUG::2014-08-11 15:54:47,226::task::595::Storage.TaskManager.Task::(_updateState) Task=`c408fd2d-9b2d-4d40-a377-5f8057b5ded0`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:54:47,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-132::INFO::2014-08-11 15:54:47,230::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:54:47,231::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:47,233::__init__::232::IOProcess::(_processLogs) (7958) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:47,231::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:47,234::__init__::232::IOProcess::(_processLogs) (7958) Queuing response >Dummy-33::DEBUG::2014-08-11 15:54:47,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-132::INFO::2014-08-11 15:54:47,262::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:54:47,264::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:47,266::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:47,266::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:47,269::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-132::DEBUG::2014-08-11 15:54:47,268::task::1191::Storage.TaskManager.Task::(prepare) Task=`c408fd2d-9b2d-4d40-a377-5f8057b5ded0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:54:47,271::__init__::232::IOProcess::(_processLogs) (7959) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:47,272::__init__::232::IOProcess::(_processLogs) (7959) Queuing response >Thread-132::DEBUG::2014-08-11 15:54:47,273::task::595::Storage.TaskManager.Task::(_updateState) Task=`c408fd2d-9b2d-4d40-a377-5f8057b5ded0`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:54:47,274::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-132::DEBUG::2014-08-11 15:54:47,274::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:54:47,276::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 15:54:47,276::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-132::DEBUG::2014-08-11 15:54:47,277::task::993::Storage.TaskManager.Task::(_decref) Task=`c408fd2d-9b2d-4d40-a377-5f8057b5ded0`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:54:47,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:47,282::__init__::232::IOProcess::(_processLogs) (7960) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:47,284::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|(7960) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:47,286::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:47,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:47,289::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:47,290::__init__::232::IOProcess::(_processLogs) (7961) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:47,291::__init__::232::IOProcess::(_processLogs) (7961) Queuing response >Thread-34::DEBUG::2014-08-11 15:54:47,292::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Dummy-33::DEBUG::2014-08-11 15:54:47,297::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0328412 s, 31.2 MB/s\n'; <rc> = 0 >Thread-34::DEBUG::2014-08-11 15:54:47,301::task::595::Storage.TaskManager.Task::(_updateState) Task=`923fc038-7fde-41a3-83dc-e7220f1d56e6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:54:47,303::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:54:47,308::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d83a02a-a176-4e6a-8389-4f82c3f9a742`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:54:47,309::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:54:47,311::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:54:47,312::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4d83a02a-a176-4e6a-8389-4f82c3f9a742`::Granted request >Thread-34::DEBUG::2014-08-11 15:54:47,314::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`923fc038-7fde-41a3-83dc-e7220f1d56e6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:54:47,315::task::993::Storage.TaskManager.Task::(_decref) Task=`923fc038-7fde-41a3-83dc-e7220f1d56e6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:54:47,362::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945137664', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:47,363::task::1191::Storage.TaskManager.Task::(prepare) Task=`923fc038-7fde-41a3-83dc-e7220f1d56e6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945137664', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:47,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`923fc038-7fde-41a3-83dc-e7220f1d56e6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:54:47,365::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:54:47,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:54:47,368::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:54:47,370::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:54:47,371::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:54:47,373::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:54:47,389::task::993::Storage.TaskManager.Task::(_decref) Task=`923fc038-7fde-41a3-83dc-e7220f1d56e6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:54:49,400::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:49,434::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132585 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:51,457::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:51,490::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134021 s, 76.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:52,276::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:52,278::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:54:52,285::task::595::Storage.TaskManager.Task::(_updateState) Task=`f5704e6f-9ae8-478f-b5d9-3fd26e320b11`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:54:52,286::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-08-11 15:54:52,289::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000862986', 'lastCheck': '5.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:54:52,290::task::1191::Storage.TaskManager.Task::(prepare) Task=`f5704e6f-9ae8-478f-b5d9-3fd26e320b11`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000862986', 'lastCheck': '5.0', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:52,290::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:54:52,291::task::595::Storage.TaskManager.Task::(_updateState) Task=`f5704e6f-9ae8-478f-b5d9-3fd26e320b11`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:54:52,292::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:54:52,294::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:54:52,295::task::993::Storage.TaskManager.Task::(_decref) Task=`f5704e6f-9ae8-478f-b5d9-3fd26e320b11`::ref 0 aborting False >Thread-132::DEBUG::2014-08-11 15:54:52,315::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:54:53,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:53,548::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136218 s, 75.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:55,571::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:55,605::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132407 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:54:57,288::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:54:57,288::__init__::232::IOProcess::(_processLogs) (7951) Queuing responseDEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:57,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:57,387::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:54:57,388::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:54:57,390::__init__::232::IOProcess::(_processLogs) (7962) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:57,410::__init__::232::IOProcess::(_processLogs) (7962) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:57,412::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:54:57,413::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-16::DEBUG::2014-08-11 15:54:57,414::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00109031 s, 417 kB/s\n'; <rc> = 0 >Thread-132::DEBUG::2014-08-11 15:54:57,417::task::595::Storage.TaskManager.Task::(_updateState) Task=`77dbfe2f-44ff-405f-a447-f033f40b2e37`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:54:57,420::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:54:57,421::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:54:57,422::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:57,423::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:57,424::__init__::232::IOProcess::(_processLogs) (7963) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:54:57,424::__init__::232::IOProcess::(_processLogs) (7963) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:57,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:57,428::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:57,429::__init__::232::IOProcess::(_processLogs) (7964) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:57,430::__init__::232::IOProcess::(_processLogs) (7964) Queuing response >Thread-14::DEBUG::2014-08-11 15:54:57,432::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:54:57,433::__init__::232::IOProcess::(_processLogs) (7965) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:54:57,434::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:54:57,435::__init__::232::IOProcess::(_processLogs) (7966) Got request for method 'access' >Thread-132::INFO::2014-08-11 15:54:57,436::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:54:57,437::__init__::232::IOProcess::(_processLogs) (7966) Queuing response >Thread-132::DEBUG::2014-08-11 15:54:57,438::task::1191::Storage.TaskManager.Task::(prepare) Task=`77dbfe2f-44ff-405f-a447-f033f40b2e37`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:54:57,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`77dbfe2f-44ff-405f-a447-f033f40b2e37`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:54:57,438::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:54:57,438::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:54:57,439::task::993::Storage.TaskManager.Task::(_decref) Task=`77dbfe2f-44ff-405f-a447-f033f40b2e37`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:54:57,449::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:54:57,450::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a704f50-745e-432f-a40a-e14cad133c32`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:54:57,450::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:54:57,453::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`508abfc2-b135-4e0a-9245-bb34e42a4166`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:54:57,453::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:54:57,453::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:54:57,454::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`508abfc2-b135-4e0a-9245-bb34e42a4166`::Granted request >Thread-34::DEBUG::2014-08-11 15:54:57,454::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9a704f50-745e-432f-a40a-e14cad133c32`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:54:57,454::task::993::Storage.TaskManager.Task::(_decref) Task=`9a704f50-745e-432f-a40a-e14cad133c32`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:54:57,474::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:57,475::task::1191::Storage.TaskManager.Task::(prepare) Task=`9a704f50-745e-432f-a40a-e14cad133c32`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:54:57,475::task::595::Storage.TaskManager.Task::(_updateState) Task=`9a704f50-745e-432f-a40a-e14cad133c32`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:54:57,475::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:54:57,475::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:54:57,476::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:54:57,476::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:54:57,476::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:54:57,476::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:54:57,477::task::993::Storage.TaskManager.Task::(_decref) Task=`9a704f50-745e-432f-a40a-e14cad133c32`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:54:57,628::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:57,666::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0164315 s, 62.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:54:59,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:54:59,723::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134297 s, 76.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:01,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:01,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134364 s, 76.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:02,439::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:02,444::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:02,449::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:55:03,802::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:03,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131776 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:05,858::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:05,891::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132454 s, 77.3 MB/s\n'; <rc> = 0 >Thread-132::DEBUG::2014-08-11 15:55:07,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`dba91ff6-f835-4630-b757-9b9578134a03`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:55:07,408::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-08-11 15:55:07,408::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00109031', 'lastCheck': '10.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:07,408::task::1191::Storage.TaskManager.Task::(prepare) Task=`dba91ff6-f835-4630-b757-9b9578134a03`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00109031', 'lastCheck': '10.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:07,409::task::595::Storage.TaskManager.Task::(_updateState) Task=`dba91ff6-f835-4630-b757-9b9578134a03`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:07,409::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:07,409::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:07,409::task::993::Storage.TaskManager.Task::(_decref) Task=`dba91ff6-f835-4630-b757-9b9578134a03`::ref 0 aborting False >Thread-132::DEBUG::2014-08-11 15:55:07,429::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:07,437::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:07,438::__init__::232::IOProcess::(_processLogs) (7967) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:55:07,439::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:55:07,458::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000708024 s, 643 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:55:07,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:07,462::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:07,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:07,464::__init__::232::IOProcess::(_processLogs) (7968) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:07,468::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:07,469::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:07,470::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:07,472::__init__::232::IOProcess::(_processLogs) (7969) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:07,474::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:07,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:07,476::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:07,477::__init__::232::IOProcess::(_processLogs) (7970) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:07,479::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:07,479::__init__::232::IOProcess::(_processLogs) (7971) Got request for method 'access' >Thread-132::DEBUG::2014-08-11 15:55:07,489::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-132::DEBUG::2014-08-11 15:55:07,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccadbcd4-9e21-4eac-8aa9-e54ebe1160f8`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:55:07,490::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-132::INFO::2014-08-11 15:55:07,503::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:07,503::task::1191::Storage.TaskManager.Task::(prepare) Task=`ccadbcd4-9e21-4eac-8aa9-e54ebe1160f8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:07,504::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccadbcd4-9e21-4eac-8aa9-e54ebe1160f8`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:07,504::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:07,504::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:07,504::task::993::Storage.TaskManager.Task::(_decref) Task=`ccadbcd4-9e21-4eac-8aa9-e54ebe1160f8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:55:07,514::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:55:07,515::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dbcb219-1f55-4f19-bdf4-c9b9ae33a76e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:55:07,515::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:55:07,516::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e6e85ed1-a36b-451d-8624-df002d87a2bc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:55:07,516::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:55:07,517::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:55:07,517::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e6e85ed1-a36b-451d-8624-df002d87a2bc`::Granted request >Thread-34::DEBUG::2014-08-11 15:55:07,517::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`9dbcb219-1f55-4f19-bdf4-c9b9ae33a76e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:55:07,518::task::993::Storage.TaskManager.Task::(_decref) Task=`9dbcb219-1f55-4f19-bdf4-c9b9ae33a76e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:55:07,537::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:07,538::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dbcb219-1f55-4f19-bdf4-c9b9ae33a76e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:07,538::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dbcb219-1f55-4f19-bdf4-c9b9ae33a76e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:55:07,538::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:55:07,538::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:55:07,539::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:55:07,539::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:55:07,539::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:55:07,539::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:55:07,540::task::993::Storage.TaskManager.Task::(_decref) Task=`9dbcb219-1f55-4f19-bdf4-c9b9ae33a76e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:55:07,914::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:07,948::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013355 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:09,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:10,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133202 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:12,027::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:12,060::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132654 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:12,484::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:12,485::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:12,487::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:55:14,083::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:14,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131586 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:16,138::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:16,172::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132482 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:55:17,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:17,482::__init__::232::IOProcess::(_processLogs) (7972) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:55:17,483::__init__::232::IOProcess::(_processLogs) DEBUG|(7972) Queuing response >Thread-16::DEBUG::2014-08-11 15:55:17,483::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:17,503::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:17,505::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-16::DEBUG::2014-08-11 15:55:17,507::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000427217 s, 1.1 MB/s\n'; <rc> = 0 >Thread-132::DEBUG::2014-08-11 15:55:17,553::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 15:55:17,590::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-132::DEBUG::2014-08-11 15:55:17,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec4ae844-64f4-4947-b70c-cde1f5d82f4c`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:55:17,592::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-132::INFO::2014-08-11 15:55:17,594::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:55:17,594::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:17,595::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:17,596::__init__::232::IOProcess::(_processLogs) (7973) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:55:17,597::__init__::232::IOProcess::(_processLogs) DEBUG|(7973) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:17,600::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:17,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:17,602::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:17,603::__init__::232::IOProcess::(_processLogs) (7974) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:17,604::__init__::232::IOProcess::(_processLogs) (7974) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:17,605::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:17,605::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:17,606::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:17,606::__init__::232::IOProcess::(_processLogs) (7975) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:17,607::__init__::232::IOProcess::(_processLogs) DEBUG|(7975) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:17,608::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:17,609::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-132::INFO::2014-08-11 15:55:17,609::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:55:17,610::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-132::DEBUG::2014-08-11 15:55:17,612::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec4ae844-64f4-4947-b70c-cde1f5d82f4c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:55:17,612::__init__::232::IOProcess::(_processLogs) (7976) Got request for method 'access' >Thread-132::DEBUG::2014-08-11 15:55:17,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec4ae844-64f4-4947-b70c-cde1f5d82f4c`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:55:17,613::__init__::232::IOProcess::(_processLogs) (7976) Queuing response >Thread-132::DEBUG::2014-08-11 15:55:17,614::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:17,615::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:17,615::task::993::Storage.TaskManager.Task::(_decref) Task=`ec4ae844-64f4-4947-b70c-cde1f5d82f4c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:55:17,626::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:55:17,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`599f871a-755e-449d-902c-6d3054d25a20`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:55:17,627::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:55:17,627::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f995a535-e58c-4924-9094-bee0ecccbbb4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:55:17,628::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:55:17,628::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:55:17,628::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f995a535-e58c-4924-9094-bee0ecccbbb4`::Granted request >Thread-34::DEBUG::2014-08-11 15:55:17,629::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`599f871a-755e-449d-902c-6d3054d25a20`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:55:17,629::task::993::Storage.TaskManager.Task::(_decref) Task=`599f871a-755e-449d-902c-6d3054d25a20`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:55:17,649::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:17,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`599f871a-755e-449d-902c-6d3054d25a20`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:17,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`599f871a-755e-449d-902c-6d3054d25a20`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:55:17,650::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:55:17,650::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:55:17,650::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:55:17,650::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:55:17,651::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:55:17,651::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:55:17,651::task::993::Storage.TaskManager.Task::(_decref) Task=`599f871a-755e-449d-902c-6d3054d25a20`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:55:18,195::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:18,229::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133301 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:20,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:20,285::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133041 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:22,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:22,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131971 s, 77.6 MB/s\n'; <rc> = 0 >Thread-132::DEBUG::2014-08-11 15:55:22,562::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f1ae4fd-01c5-4c8e-b601-ac6d694ec57e`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:55:22,562::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-08-11 15:55:22,587::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427217', 'lastCheck': '5.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:22,587::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f1ae4fd-01c5-4c8e-b601-ac6d694ec57e`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000427217', 'lastCheck': '5.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:22,587::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f1ae4fd-01c5-4c8e-b601-ac6d694ec57e`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:22,587::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:22,587::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:22,588::task::993::Storage.TaskManager.Task::(_decref) Task=`1f1ae4fd-01c5-4c8e-b601-ac6d694ec57e`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:22,610::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:22,612::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:22,613::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:55:22,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:55:24,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:24,398::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013351 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:26,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:26,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013435 s, 76.2 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:55:26,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`c73ddb01-1770-443a-8d46-a9d1c9911c81`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:55:26,939::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:55:26,940::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:26,941::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:26,941::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:26,941::__init__::232::IOProcess::(_processLogs) (7977) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:55:26,942::__init__::232::IOProcess::(_processLogs) DEBUG|(7977) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:26,944::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:26,944::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:26,944::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:26,944::__init__::232::IOProcess::(_processLogs) (7978) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:55:26,945::__init__::232::IOProcess::(_processLogs) (7978) Queuing response >Thread-122::INFO::2014-08-11 15:55:26,945::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:55:26,945::task::1191::Storage.TaskManager.Task::(prepare) Task=`c73ddb01-1770-443a-8d46-a9d1c9911c81`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:55:26,946::task::595::Storage.TaskManager.Task::(_updateState) Task=`c73ddb01-1770-443a-8d46-a9d1c9911c81`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:55:26,946::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:55:26,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:55:26,946::task::993::Storage.TaskManager.Task::(_decref) Task=`c73ddb01-1770-443a-8d46-a9d1c9911c81`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:55:27,612::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:27,613::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:27,619::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:55:27,663::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 15:55:27,690::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-132::DEBUG::2014-08-11 15:55:27,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`619be369-b7dc-45e9-9242-32e3d451c670`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:55:27,693::__init__::232::IOProcess::(_processLogs) (7979) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:55:27,693::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-132::INFO::2014-08-11 15:55:27,695::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:27,696::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:27,697::__init__::232::IOProcess::(_processLogs) (7979) Queuing response >Thread-16::DEBUG::2014-08-11 15:55:27,719::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00158673 s, 287 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:55:27,722::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:27,723::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:27,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:27,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:27,726::__init__::232::IOProcess::(_processLogs) (7980) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:55:27,731::__init__::232::IOProcess::(_processLogs) DEBUG|(7980) Queuing response >Thread-132::INFO::2014-08-11 15:55:27,731::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:27,733::task::1191::Storage.TaskManager.Task::(prepare) Task=`619be369-b7dc-45e9-9242-32e3d451c670`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:27,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`619be369-b7dc-45e9-9242-32e3d451c670`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:27,736::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:55:27,735::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-132::DEBUG::2014-08-11 15:55:27,737::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-14::DEBUG::2014-08-11 15:55:27,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-132::DEBUG::2014-08-11 15:55:27,738::task::993::Storage.TaskManager.Task::(_decref) Task=`619be369-b7dc-45e9-9242-32e3d451c670`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:55:27,739::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:27,741::__init__::232::IOProcess::(_processLogs) (7981) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:27,741::__init__::232::IOProcess::(_processLogs) (7981) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:27,742::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:27,742::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:27,742::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:27,743::__init__::232::IOProcess::(_processLogs) DEBUG|(7982) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:27,743::__init__::232::IOProcess::(_processLogs) (7982) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:27,744::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:27,745::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:27,746::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:27,746::__init__::232::IOProcess::(_processLogs) (7983) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:27,746::__init__::232::IOProcess::(_processLogs) (7983) Queuing response >Thread-34::DEBUG::2014-08-11 15:55:27,751::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:55:27,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`960e5a3d-3bf8-4f6b-901d-1cd8bd7a6c02`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:55:27,752::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:55:27,753::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2330393c-3aaa-410f-9c0c-6d44c1847d51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:55:27,753::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:55:27,754::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:55:27,754::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2330393c-3aaa-410f-9c0c-6d44c1847d51`::Granted request >Thread-34::DEBUG::2014-08-11 15:55:27,754::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`960e5a3d-3bf8-4f6b-901d-1cd8bd7a6c02`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:55:27,755::task::993::Storage.TaskManager.Task::(_decref) Task=`960e5a3d-3bf8-4f6b-901d-1cd8bd7a6c02`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:55:27,774::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:27,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`960e5a3d-3bf8-4f6b-901d-1cd8bd7a6c02`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:27,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`960e5a3d-3bf8-4f6b-901d-1cd8bd7a6c02`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:55:27,775::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:55:27,775::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:55:27,776::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:55:27,776::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:55:27,776::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:55:27,777::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:55:27,777::task::993::Storage.TaskManager.Task::(_decref) Task=`960e5a3d-3bf8-4f6b-901d-1cd8bd7a6c02`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:55:28,478::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:28,512::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132399 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:30,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:30,569::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133775 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:32,592::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:32,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0268301 s, 38.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:32,738::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:32,750::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:32,761::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:55:34,876::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:34,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132874 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:36,933::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:36,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135325 s, 75.7 MB/s\n'; <rc> = 0 >Thread-132::DEBUG::2014-08-11 15:55:37,720::task::595::Storage.TaskManager.Task::(_updateState) Task=`09dd0348-319d-46ec-8303-244dff2ede98`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:55:37,720::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-08-11 15:55:37,721::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00158673', 'lastCheck': '10.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:37,721::task::1191::Storage.TaskManager.Task::(prepare) Task=`09dd0348-319d-46ec-8303-244dff2ede98`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00158673', 'lastCheck': '10.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:37,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`09dd0348-319d-46ec-8303-244dff2ede98`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:37,722::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:37,722::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:37,722::task::993::Storage.TaskManager.Task::(_decref) Task=`09dd0348-319d-46ec-8303-244dff2ede98`::ref 0 aborting False >Thread-132::DEBUG::2014-08-11 15:55:37,743::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:37,747::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:37,747::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:37,747::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:37,748::__init__::232::IOProcess::(_processLogs) (7984) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:55:37,748::__init__::232::IOProcess::(_processLogs) (7984) Queuing response >Thread-16::DEBUG::2014-08-11 15:55:37,749::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:55:37,772::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000422972 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:55:37,774::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:55:37,774::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:37,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:37,775::__init__::232::IOProcess::(_processLogs) (7985) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:55:37,778::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:37,778::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:37,779::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:37,781::__init__::232::IOProcess::(_processLogs) (7986) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:37,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:37,785::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:37,786::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:37,788::__init__::232::IOProcess::(_processLogs) (7987) Got request for method 'access' >Thread-132::DEBUG::2014-08-11 15:55:37,790::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-14::DEBUG::2014-08-11 15:55:37,792::__init__::232::IOProcess::(_processLogs) (7987) Queuing response >Thread-132::DEBUG::2014-08-11 15:55:37,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c439d06-dc90-487b-957b-aaa847f9c8d6`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:37,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:37,795::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-132::INFO::2014-08-11 15:55:37,796::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:55:37,797::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:37,799::__init__::232::IOProcess::(_processLogs) (7988) Got request for method 'access' >Thread-132::INFO::2014-08-11 15:55:37,811::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:37,812::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c439d06-dc90-487b-957b-aaa847f9c8d6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:37,812::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c439d06-dc90-487b-957b-aaa847f9c8d6`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:37,812::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:37,812::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:37,813::task::993::Storage.TaskManager.Task::(_decref) Task=`9c439d06-dc90-487b-957b-aaa847f9c8d6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:55:37,823::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:55:37,823::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7f5b2e1-1c51-4ffe-b214-89c349aaa979`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:55:37,824::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:55:37,824::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f5c50c4f-60cb-4ae8-ba52-71eb7f9afd9e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:55:37,825::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:55:37,825::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:55:37,825::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f5c50c4f-60cb-4ae8-ba52-71eb7f9afd9e`::Granted request >Thread-34::DEBUG::2014-08-11 15:55:37,826::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d7f5b2e1-1c51-4ffe-b214-89c349aaa979`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:55:37,826::task::993::Storage.TaskManager.Task::(_decref) Task=`d7f5b2e1-1c51-4ffe-b214-89c349aaa979`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:55:37,846::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:37,846::task::1191::Storage.TaskManager.Task::(prepare) Task=`d7f5b2e1-1c51-4ffe-b214-89c349aaa979`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:37,846::task::595::Storage.TaskManager.Task::(_updateState) Task=`d7f5b2e1-1c51-4ffe-b214-89c349aaa979`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:55:37,847::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:55:37,847::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:55:37,847::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:55:37,848::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:55:37,848::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:55:37,848::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:55:37,848::task::993::Storage.TaskManager.Task::(_decref) Task=`d7f5b2e1-1c51-4ffe-b214-89c349aaa979`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:55:38,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:39,025::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131656 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:41,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:41,081::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132065 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:42,807::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:42,889::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:42,890::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:55:43,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:43,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132628 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:45,160::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:45,194::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133011 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:47,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:47,251::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131255 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:55:47,802::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:47,802::__init__::232::IOProcess::(_processLogs) (7989) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:55:47,803::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:55:47,823::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000432643 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:55:47,824::__init__::232::IOProcess::(_processLogs) (7990) Got request for method 'statvfs' >Thread-132::DEBUG::2014-08-11 15:55:47,862::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-132::DEBUG::2014-08-11 15:55:47,891::task::595::Storage.TaskManager.Task::(_updateState) Task=`65d691b7-a0cd-4d80-a625-8e4a5b66df80`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:55:47,892::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:55:47,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:47,893::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:47,894::__init__::232::IOProcess::(_processLogs) (7991) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:47,895::__init__::232::IOProcess::(_processLogs) (7992) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:47,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:47,901::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:55:47,904::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:47,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::INFO::2014-08-11 15:55:47,907::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:55:47,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-132::DEBUG::2014-08-11 15:55:47,909::task::1191::Storage.TaskManager.Task::(prepare) Task=`65d691b7-a0cd-4d80-a625-8e4a5b66df80`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-14::DEBUG::2014-08-11 15:55:47,909::__init__::232::IOProcess::(_processLogs) (7993) Got request for method 'access' >Thread-132::DEBUG::2014-08-11 15:55:47,910::task::595::Storage.TaskManager.Task::(_updateState) Task=`65d691b7-a0cd-4d80-a625-8e4a5b66df80`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:47,911::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-14::DEBUG::2014-08-11 15:55:47,911::__init__::232::IOProcess::(_processLogs) (7993) Queuing response >Thread-132::DEBUG::2014-08-11 15:55:47,913::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:47,913::task::993::Storage.TaskManager.Task::(_decref) Task=`65d691b7-a0cd-4d80-a625-8e4a5b66df80`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:55:47,927::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:55:47,927::task::595::Storage.TaskManager.Task::(_updateState) Task=`70f04593-308e-4159-a334-0d12e71717f6`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:55:47,928::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:55:47,928::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`334d1c66-2eb4-4449-a118-97abe9bb94f3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:55:47,928::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:55:47,929::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:55:47,929::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`334d1c66-2eb4-4449-a118-97abe9bb94f3`::Granted request >Thread-34::DEBUG::2014-08-11 15:55:47,930::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`70f04593-308e-4159-a334-0d12e71717f6`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:55:47,930::task::993::Storage.TaskManager.Task::(_decref) Task=`70f04593-308e-4159-a334-0d12e71717f6`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:55:47,949::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945072128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:47,950::task::1191::Storage.TaskManager.Task::(prepare) Task=`70f04593-308e-4159-a334-0d12e71717f6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945072128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:47,950::task::595::Storage.TaskManager.Task::(_updateState) Task=`70f04593-308e-4159-a334-0d12e71717f6`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:55:47,950::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:55:47,951::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:55:47,951::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:55:47,951::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:55:47,951::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:55:47,952::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:55:47,952::task::993::Storage.TaskManager.Task::(_decref) Task=`70f04593-308e-4159-a334-0d12e71717f6`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:55:49,275::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:49,309::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013381 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:51,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:51,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133392 s, 76.8 MB/s\n'; <rc> = 0 >Thread-132::DEBUG::2014-08-11 15:55:52,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e4921c9-aa82-4c5a-9595-2febce69f532`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:55:52,843::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-08-11 15:55:52,890::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432643', 'lastCheck': '5.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:52,890::task::1191::Storage.TaskManager.Task::(prepare) Task=`1e4921c9-aa82-4c5a-9595-2febce69f532`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000432643', 'lastCheck': '5.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:55:52,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`1e4921c9-aa82-4c5a-9595-2febce69f532`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:52,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:52,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:52,891::task::993::Storage.TaskManager.Task::(_decref) Task=`1e4921c9-aa82-4c5a-9595-2febce69f532`::ref 0 aborting False >Thread-132::DEBUG::2014-08-11 15:55:52,910::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:52,918::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:52,919::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:52,920::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:55:53,388::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:53,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131517 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:55:55,445::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:57,408::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 1.94238 s, 527 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:55:57,907::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:57,907::__init__::232::IOProcess::(_processLogs) (7994) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:55:57,908::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:57,925::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:55:57,965::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:58,011::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:55:58,013::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-132::DEBUG::2014-08-11 15:55:58,015::task::595::Storage.TaskManager.Task::(_updateState) Task=`5dc2cf78-b984-4ff7-8980-9350dc1f73ae`::moving from state init -> state preparing >Thread-16::DEBUG::2014-08-11 15:55:58,016::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00451928 s, 101 kB/s\n'; <rc> = 0 >Thread-132::INFO::2014-08-11 15:55:58,016::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:55:58,018::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:58,019::__init__::232::IOProcess::(_processLogs) (7995) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:55:58,023::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:58,023::__init__::232::IOProcess::(_processLogs) (7996) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:58,024::__init__::232::IOProcess::(_processLogs) (7996) Queuing response >Thread-14::DEBUG::2014-08-11 15:55:58,024::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:58,025::__init__::232::IOProcess::(_processLogs) (7997) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:58,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:55:58,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:55:58,029::__init__::232::IOProcess::(_processLogs) (7998) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:55:58,029::__init__::232::IOProcess::(_processLogs) (7998) Queuing response >Thread-132::INFO::2014-08-11 15:55:58,031::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:58,031::task::1191::Storage.TaskManager.Task::(prepare) Task=`5dc2cf78-b984-4ff7-8980-9350dc1f73ae`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-132::DEBUG::2014-08-11 15:55:58,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`5dc2cf78-b984-4ff7-8980-9350dc1f73ae`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:55:58,032::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:55:58,032::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:55:58,032::task::993::Storage.TaskManager.Task::(_decref) Task=`5dc2cf78-b984-4ff7-8980-9350dc1f73ae`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:55:58,043::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:55:58,044::task::595::Storage.TaskManager.Task::(_updateState) Task=`76af7e5c-4930-4fb8-a92a-d528bdbd8f4a`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:55:58,044::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:55:58,045::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`df3ac115-e64d-4fa7-9d0c-baa859cf8adb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:55:58,045::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:55:58,045::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:55:58,045::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`df3ac115-e64d-4fa7-9d0c-baa859cf8adb`::Granted request >Thread-34::DEBUG::2014-08-11 15:55:58,046::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`76af7e5c-4930-4fb8-a92a-d528bdbd8f4a`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:55:58,046::task::993::Storage.TaskManager.Task::(_decref) Task=`76af7e5c-4930-4fb8-a92a-d528bdbd8f4a`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:55:58,066::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:58,066::task::1191::Storage.TaskManager.Task::(prepare) Task=`76af7e5c-4930-4fb8-a92a-d528bdbd8f4a`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:55:58,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`76af7e5c-4930-4fb8-a92a-d528bdbd8f4a`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:55:58,067::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:55:58,067::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:55:58,068::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:55:58,068::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:55:58,068::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:55:58,068::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:55:58,069::task::993::Storage.TaskManager.Task::(_decref) Task=`76af7e5c-4930-4fb8-a92a-d528bdbd8f4a`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:55:59,431::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:55:59,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133358 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:01,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:01,521::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135757 s, 75.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:03,020::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:03,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:03,090::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:56:03,545::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:03,579::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132547 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:05,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:05,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132133 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:07,658::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:07,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133209 s, 76.9 MB/s\n'; <rc> = 0 >Thread-132::DEBUG::2014-08-11 15:56:08,006::task::595::Storage.TaskManager.Task::(_updateState) Task=`11a3d6ab-eb7c-419c-8ae2-4fbfaa40dd06`::moving from state init -> state preparing >Thread-132::INFO::2014-08-11 15:56:08,006::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-132::INFO::2014-08-11 15:56:08,007::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00451928', 'lastCheck': '10.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:56:08,007::task::1191::Storage.TaskManager.Task::(prepare) Task=`11a3d6ab-eb7c-419c-8ae2-4fbfaa40dd06`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00451928', 'lastCheck': '10.0', 'valid': True}} >Thread-132::DEBUG::2014-08-11 15:56:08,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`11a3d6ab-eb7c-419c-8ae2-4fbfaa40dd06`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-08-11 15:56:08,007::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-08-11 15:56:08,008::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-08-11 15:56:08,008::task::993::Storage.TaskManager.Task::(_decref) Task=`11a3d6ab-eb7c-419c-8ae2-4fbfaa40dd06`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:56:08,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:08,029::__init__::232::IOProcess::(_processLogs) (7999) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:56:08,032::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-132::DEBUG::2014-08-11 15:56:08,033::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 15:56:08,109::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:43319 >Thread-16::DEBUG::2014-08-11 15:56:08,114::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000776539 s, 586 kB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:08,115::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 15:56:08,116::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:43319 >Thread-14::DEBUG::2014-08-11 15:56:08,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Detector thread::DEBUG::2014-08-11 15:56:08,120::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:43319 >Thread-14::DEBUG::2014-08-11 15:56:08,122::__init__::232::IOProcess::(_processLogs) Extracting request information... >Detector thread::DEBUG::2014-08-11 15:56:08,125::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 43319) >Thread-14::DEBUG::2014-08-11 15:56:08,126::__init__::232::IOProcess::(_processLogs) (8000) Got request for method 'statvfs' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:08,129::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:56:08,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-133::DEBUG::2014-08-11 15:56:08,140::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-14::DEBUG::2014-08-11 15:56:08,143::__init__::232::IOProcess::(_processLogs) Extracting request information... >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:08,145::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:08,146::task::595::Storage.TaskManager.Task::(_updateState) Task=`d07fa882-b6bd-4c69-9aa3-adf35dee9e85`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:56:08,146::__init__::232::IOProcess::(_processLogs) (8001) Got request for method 'access' >Thread-133::INFO::2014-08-11 15:56:08,150::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:56:08,151::__init__::232::IOProcess::(_processLogs) (8001) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:08,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:08,155::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:08,157::__init__::232::IOProcess::(_processLogs) (8002) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:08,157::__init__::232::IOProcess::(_processLogs) (8002) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:08,157::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:08,158::__init__::232::IOProcess::(_processLogs) (8003) Queuing response >Thread-133::INFO::2014-08-11 15:56:08,169::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:08,170::task::1191::Storage.TaskManager.Task::(prepare) Task=`d07fa882-b6bd-4c69-9aa3-adf35dee9e85`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:08,170::task::595::Storage.TaskManager.Task::(_updateState) Task=`d07fa882-b6bd-4c69-9aa3-adf35dee9e85`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:08,170::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:08,170::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:08,171::task::993::Storage.TaskManager.Task::(_decref) Task=`d07fa882-b6bd-4c69-9aa3-adf35dee9e85`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:56:08,181::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [4e931c41] >Thread-34::DEBUG::2014-08-11 15:56:08,182::task::595::Storage.TaskManager.Task::(_updateState) Task=`e190a7c2-35d1-4604-919c-eaf851d86606`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:56:08,182::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:56:08,183::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6d06523d-fd8c-421f-8438-a416c171221e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:56:08,183::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:56:08,184::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:56:08,184::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6d06523d-fd8c-421f-8438-a416c171221e`::Granted request >Thread-34::DEBUG::2014-08-11 15:56:08,184::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e190a7c2-35d1-4604-919c-eaf851d86606`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:56:08,185::task::993::Storage.TaskManager.Task::(_decref) Task=`e190a7c2-35d1-4604-919c-eaf851d86606`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:56:08,205::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:08,205::task::1191::Storage.TaskManager.Task::(prepare) Task=`e190a7c2-35d1-4604-919c-eaf851d86606`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:08,205::task::595::Storage.TaskManager.Task::(_updateState) Task=`e190a7c2-35d1-4604-919c-eaf851d86606`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:56:08,205::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:56:08,206::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:56:08,206::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:56:08,206::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:56:08,207::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:56:08,207::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:56:08,207::task::993::Storage.TaskManager.Task::(_decref) Task=`e190a7c2-35d1-4604-919c-eaf851d86606`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:56:09,715::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:09,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133909 s, 76.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:11,772::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:11,806::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131106 s, 78.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:13,161::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:13,162::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:13,190::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:56:13,829::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:13,862::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132172 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:15,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:15,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132336 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:17,942::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:17,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134506 s, 76.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:18,160::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:18,161::__init__::232::IOProcess::(_processLogs) (8004) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:56:18,161::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:56:18,181::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424636 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:18,182::__init__::232::IOProcess::(_processLogs) (8005) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:56:18,190::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:18,191::__init__::232::IOProcess::(_processLogs) (8006) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:18,192::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:18,192::__init__::232::IOProcess::(_processLogs) (8007) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:18,193::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:18,193::__init__::232::IOProcess::(_processLogs) (8008) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:18,197::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:18,198::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:18,199::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:18,220::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:56:18,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`55f9c238-416e-4fbf-8696-a1d171591896`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:56:18,221::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-133::INFO::2014-08-11 15:56:18,234::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:18,234::task::1191::Storage.TaskManager.Task::(prepare) Task=`55f9c238-416e-4fbf-8696-a1d171591896`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:18,235::task::595::Storage.TaskManager.Task::(_updateState) Task=`55f9c238-416e-4fbf-8696-a1d171591896`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:18,235::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:18,235::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:18,235::task::993::Storage.TaskManager.Task::(_decref) Task=`55f9c238-416e-4fbf-8696-a1d171591896`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:56:18,246::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:56:18,247::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ceb72c1-e9c0-40eb-b523-c278baf3fe57`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:56:18,247::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:56:18,248::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d37a31c-f4ed-4020-815f-4f67dca1bd18`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:56:18,248::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:56:18,249::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:56:18,249::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`5d37a31c-f4ed-4020-815f-4f67dca1bd18`::Granted request >Thread-34::DEBUG::2014-08-11 15:56:18,249::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4ceb72c1-e9c0-40eb-b523-c278baf3fe57`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:56:18,250::task::993::Storage.TaskManager.Task::(_decref) Task=`4ceb72c1-e9c0-40eb-b523-c278baf3fe57`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:56:18,275::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:18,275::task::1191::Storage.TaskManager.Task::(prepare) Task=`4ceb72c1-e9c0-40eb-b523-c278baf3fe57`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:18,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`4ceb72c1-e9c0-40eb-b523-c278baf3fe57`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:56:18,276::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:56:18,276::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:56:18,276::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:56:18,277::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:56:18,277::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:56:18,277::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:56:18,277::task::993::Storage.TaskManager.Task::(_decref) Task=`4ceb72c1-e9c0-40eb-b523-c278baf3fe57`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:56:19,999::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:20,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013147 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:22,056::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:22,089::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131168 s, 78.1 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:23,206::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:23,207::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:23,254::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bff0ce4-554d-4e6e-87e6-6021c9ccfa39`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:56:23,291::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 15:56:23,293::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424636', 'lastCheck': '5.1', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:23,294::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:23,294::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bff0ce4-554d-4e6e-87e6-6021c9ccfa39`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424636', 'lastCheck': '5.1', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:56:23,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bff0ce4-554d-4e6e-87e6-6021c9ccfa39`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:23,297::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:23,298::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:23,298::task::993::Storage.TaskManager.Task::(_decref) Task=`4bff0ce4-554d-4e6e-87e6-6021c9ccfa39`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 15:56:23,316::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:56:24,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:24,146::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134992 s, 75.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:26,169::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:26,202::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133101 s, 76.9 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:56:27,061::task::595::Storage.TaskManager.Task::(_updateState) Task=`be943ec6-2939-479f-bbed-e5c75f2a8334`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:56:27,061::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:56:27,062::__init__::232::IOProcess::(_processLogs) (8009) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:56:27,065::__init__::232::IOProcess::(_processLogs) (8010) Got request for method 'stat' >Thread-122::INFO::2014-08-11 15:56:27,065::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:56:27,065::task::1191::Storage.TaskManager.Task::(prepare) Task=`be943ec6-2939-479f-bbed-e5c75f2a8334`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:56:27,066::task::595::Storage.TaskManager.Task::(_updateState) Task=`be943ec6-2939-479f-bbed-e5c75f2a8334`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:56:27,066::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:56:27,066::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:56:27,066::task::993::Storage.TaskManager.Task::(_decref) Task=`be943ec6-2939-479f-bbed-e5c75f2a8334`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:56:28,195::__init__::232::IOProcess::(_processLogs) (8011) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:56:28,196::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:56:28,216::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00042919 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:28,217::__init__::232::IOProcess::(_processLogs) Extracting request information... >Dummy-33::DEBUG::2014-08-11 15:56:28,225::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-133::DEBUG::2014-08-11 15:56:28,291::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-133::DEBUG::2014-08-11 15:56:28,314::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d2a385d-003c-4c21-a762-55e7ebac19a7`::moving from state init -> state preparing >Thread-14::DEBUG::2014-08-11 15:56:28,312::__init__::232::IOProcess::(_processLogs) (8012) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:28,317::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:56:28,318::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-14::DEBUG::2014-08-11 15:56:28,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:28,321::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:28,322::__init__::232::IOProcess::(_processLogs) (8013) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:28,325::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:28,326::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-14::DEBUG::2014-08-11 15:56:28,328::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:28,329::__init__::232::IOProcess::(_processLogs) (8014) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:28,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:28,334::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:28,337::__init__::232::IOProcess::(_processLogs) (8015) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:28,340::__init__::232::IOProcess::(_processLogs) (8015) Queuing response >Thread-133::INFO::2014-08-11 15:56:28,344::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Dummy-33::DEBUG::2014-08-11 15:56:28,344::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0297402 s, 34.4 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 15:56:28,346::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d2a385d-003c-4c21-a762-55e7ebac19a7`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:28,346::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d2a385d-003c-4c21-a762-55e7ebac19a7`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:28,347::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:28,348::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:28,349::task::993::Storage.TaskManager.Task::(_decref) Task=`3d2a385d-003c-4c21-a762-55e7ebac19a7`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:56:28,363::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [55a07894] >Thread-34::DEBUG::2014-08-11 15:56:28,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`07f8d964-ada6-4b33-a489-ac6611a250ce`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:56:28,367::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:56:28,371::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6aa70833-da20-41f0-b302-3181c633bd3a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:56:28,372::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:56:28,374::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:56:28,375::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6aa70833-da20-41f0-b302-3181c633bd3a`::Granted request >Thread-34::DEBUG::2014-08-11 15:56:28,378::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`07f8d964-ada6-4b33-a489-ac6611a250ce`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:56:28,379::task::993::Storage.TaskManager.Task::(_decref) Task=`07f8d964-ada6-4b33-a489-ac6611a250ce`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:56:28,430::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:28,430::task::1191::Storage.TaskManager.Task::(prepare) Task=`07f8d964-ada6-4b33-a489-ac6611a250ce`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:28,430::task::595::Storage.TaskManager.Task::(_updateState) Task=`07f8d964-ada6-4b33-a489-ac6611a250ce`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:56:28,430::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:56:28,431::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:56:28,431::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:56:28,431::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:56:28,432::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:56:28,432::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:56:28,432::task::993::Storage.TaskManager.Task::(_decref) Task=`07f8d964-ada6-4b33-a489-ac6611a250ce`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:56:30,422::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:30,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013252 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:32,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:32,530::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0277434 s, 36.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:33,334::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:33,335::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:33,394::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:56:34,805::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:34,853::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0249069 s, 41.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:37,044::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:37,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131705 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:38,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:56:38,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:38,345::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:38,345::__init__::232::IOProcess::(_processLogs) (8016) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:56:38,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(8016) Queuing response >Thread-16::DEBUG::2014-08-11 15:56:38,347::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:56:38,367::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420669 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:38,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:56:38,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:38,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:38,370::__init__::232::IOProcess::(_processLogs) (8017) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:56:38,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(8017) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:38,371::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:56:38,371::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:38,372::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:38,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(8018) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:38,372::__init__::232::IOProcess::(_processLogs) (8018) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:38,373::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:56:38,373::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:38,374::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:38,374::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|(8019) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:38,374::__init__::232::IOProcess::(_processLogs) (8019) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:38,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:56:38,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:38,376::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:38,376::__init__::232::IOProcess::(_processLogs) (8020) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:38,376::__init__::232::IOProcess::(_processLogs) (8020) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:38,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:38,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:38,403::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:38,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`f102aacd-506e-4885-bdb6-e49da1d2e3b8`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:56:38,406::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 15:56:38,407::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420669', 'lastCheck': '0.0', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:56:38,407::task::1191::Storage.TaskManager.Task::(prepare) Task=`f102aacd-506e-4885-bdb6-e49da1d2e3b8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000420669', 'lastCheck': '0.0', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:56:38,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`f102aacd-506e-4885-bdb6-e49da1d2e3b8`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:38,408::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:38,408::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:38,408::task::993::Storage.TaskManager.Task::(_decref) Task=`f102aacd-506e-4885-bdb6-e49da1d2e3b8`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 15:56:38,426::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:38,444::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:56:38,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cf29085-87c8-4f7f-9645-c6b8f05ce5fb`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:56:38,445::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-133::INFO::2014-08-11 15:56:38,458::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:38,458::task::1191::Storage.TaskManager.Task::(prepare) Task=`9cf29085-87c8-4f7f-9645-c6b8f05ce5fb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:38,458::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cf29085-87c8-4f7f-9645-c6b8f05ce5fb`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:38,459::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:38,459::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:38,459::task::993::Storage.TaskManager.Task::(_decref) Task=`9cf29085-87c8-4f7f-9645-c6b8f05ce5fb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:56:38,468::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:56:38,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`e158640c-d052-4ff3-9471-cca1e7bb109d`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:56:38,469::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:56:38,470::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b61275ca-a74c-4661-b542-c9e9311fbaa7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:56:38,470::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:56:38,471::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:56:38,471::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b61275ca-a74c-4661-b542-c9e9311fbaa7`::Granted request >Thread-34::DEBUG::2014-08-11 15:56:38,473::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`e158640c-d052-4ff3-9471-cca1e7bb109d`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:56:38,474::task::993::Storage.TaskManager.Task::(_decref) Task=`e158640c-d052-4ff3-9471-cca1e7bb109d`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:56:38,493::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945072128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:38,494::task::1191::Storage.TaskManager.Task::(prepare) Task=`e158640c-d052-4ff3-9471-cca1e7bb109d`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945072128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:38,494::task::595::Storage.TaskManager.Task::(_updateState) Task=`e158640c-d052-4ff3-9471-cca1e7bb109d`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:56:38,494::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:56:38,495::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:56:38,495::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:56:38,495::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:56:38,495::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:56:38,496::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:56:38,496::task::993::Storage.TaskManager.Task::(_decref) Task=`e158640c-d052-4ff3-9471-cca1e7bb109d`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:56:39,101::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:39,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132152 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:41,157::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:41,190::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132396 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:43,213::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:43,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0160723 s, 63.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:43,410::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:43,412::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:43,494::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:56:45,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:45,307::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132121 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:47,329::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:47,364::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133422 s, 76.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:48,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:56:48,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:48,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:48,379::__init__::232::IOProcess::(_processLogs) (8021) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:56:48,379::__init__::232::IOProcess::(_processLogs) (8021) Queuing response >Thread-16::DEBUG::2014-08-11 15:56:48,380::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:56:48,400::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000436516 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:48,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:56:48,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:56:48,402::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:48,402::__init__::232::IOProcess::(_processLogs) (8022) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:56:48,403::__init__::232::IOProcess::(_processLogs) (8023) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:48,404::__init__::232::IOProcess::(_processLogs) (8023) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:48,404::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:48,405::__init__::232::IOProcess::(_processLogs) (8024) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:48,406::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:48,406::__init__::232::IOProcess::(_processLogs) (8025) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:48,406::__init__::232::IOProcess::(_processLogs) (8025) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:48,498::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:48,499::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:48,509::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:56:48,596::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e1e662f-3d0a-4fa6-a675-c99fc7cc2d82`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:56:48,598::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:48,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:56:48,612::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:48,613::task::1191::Storage.TaskManager.Task::(prepare) Task=`9e1e662f-3d0a-4fa6-a675-c99fc7cc2d82`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:48,613::task::595::Storage.TaskManager.Task::(_updateState) Task=`9e1e662f-3d0a-4fa6-a675-c99fc7cc2d82`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:48,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:48,613::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:48,614::task::993::Storage.TaskManager.Task::(_decref) Task=`9e1e662f-3d0a-4fa6-a675-c99fc7cc2d82`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:56:48,624::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:56:48,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`275f1fb9-18c7-4a9d-8a71-7a897fad54fd`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:56:48,625::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:56:48,626::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9891c77b-8391-4a61-8594-1487a74b7b70`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:56:48,626::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:56:48,626::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:56:48,626::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9891c77b-8391-4a61-8594-1487a74b7b70`::Granted request >Thread-34::DEBUG::2014-08-11 15:56:48,627::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`275f1fb9-18c7-4a9d-8a71-7a897fad54fd`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:56:48,627::task::993::Storage.TaskManager.Task::(_decref) Task=`275f1fb9-18c7-4a9d-8a71-7a897fad54fd`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:56:48,651::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:48,651::task::1191::Storage.TaskManager.Task::(prepare) Task=`275f1fb9-18c7-4a9d-8a71-7a897fad54fd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:48,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`275f1fb9-18c7-4a9d-8a71-7a897fad54fd`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:56:48,651::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:56:48,652::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:56:48,652::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:56:48,652::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:56:48,653::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:56:48,653::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:56:48,653::task::993::Storage.TaskManager.Task::(_decref) Task=`275f1fb9-18c7-4a9d-8a71-7a897fad54fd`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:56:49,384::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:49,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133233 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:51,440::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:51,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133733 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:53,498::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-133::DEBUG::2014-08-11 15:56:53,526::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c89a9e8-af95-40c3-9db0-9f4b32cf64a4`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:56:53,527::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Dummy-33::DEBUG::2014-08-11 15:56:53,532::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141165 s, 72.5 MB/s\n'; <rc> = 0 >Thread-133::INFO::2014-08-11 15:56:53,595::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436516', 'lastCheck': '5.2', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:56:53,596::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c89a9e8-af95-40c3-9db0-9f4b32cf64a4`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000436516', 'lastCheck': '5.2', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:56:53,597::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c89a9e8-af95-40c3-9db0-9f4b32cf64a4`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:53,598::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:53,599::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:53,600::task::993::Storage.TaskManager.Task::(_decref) Task=`9c89a9e8-af95-40c3-9db0-9f4b32cf64a4`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:53,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:53,620::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:53,630::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:53,648::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:56:55,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:55,707::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132142 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:56:57,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:57,763::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132873 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:58,408::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:58,409::__init__::232::IOProcess::(_processLogs) (8026) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:56:58,409::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:56:58,429::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426808 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:56:58,431::__init__::232::IOProcess::(_processLogs) (8027) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:56:58,432::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:58,433::__init__::232::IOProcess::(_processLogs) (8028) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:58,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:58,434::__init__::232::IOProcess::(_processLogs) (8029) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:58,434::__init__::232::IOProcess::(_processLogs) (8029) Queuing response >Thread-14::DEBUG::2014-08-11 15:56:58,435::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:56:58,435::__init__::232::IOProcess::(_processLogs) (8030) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:56:58,436::__init__::232::IOProcess::(_processLogs) (8030) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:58,640::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:58,641::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:56:58,666::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:56:58,696::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf309735-bf75-4956-a2a8-aa9536cf3412`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:56:58,698::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:56:58,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-133::INFO::2014-08-11 15:56:58,715::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:58,715::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf309735-bf75-4956-a2a8-aa9536cf3412`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:56:58,715::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf309735-bf75-4956-a2a8-aa9536cf3412`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:56:58,716::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:56:58,716::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:56:58,716::task::993::Storage.TaskManager.Task::(_decref) Task=`bf309735-bf75-4956-a2a8-aa9536cf3412`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:56:58,728::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:56:58,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2b1bb24-12c8-46db-a78e-a56b672741aa`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:56:58,729::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:56:58,730::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dbc33945-9468-4c5f-bc67-106b721d2166`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:56:58,730::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:56:58,730::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:56:58,730::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`dbc33945-9468-4c5f-bc67-106b721d2166`::Granted request >Thread-34::DEBUG::2014-08-11 15:56:58,731::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f2b1bb24-12c8-46db-a78e-a56b672741aa`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:56:58,731::task::993::Storage.TaskManager.Task::(_decref) Task=`f2b1bb24-12c8-46db-a78e-a56b672741aa`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:56:58,751::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:58,751::task::1191::Storage.TaskManager.Task::(prepare) Task=`f2b1bb24-12c8-46db-a78e-a56b672741aa`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:56:58,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2b1bb24-12c8-46db-a78e-a56b672741aa`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:56:58,752::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:56:58,752::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:56:58,753::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:56:58,753::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:56:58,753::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:56:58,753::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:56:58,754::task::993::Storage.TaskManager.Task::(_decref) Task=`f2b1bb24-12c8-46db-a78e-a56b672741aa`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:56:59,784::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:56:59,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137358 s, 74.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:01,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:01,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132726 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:03,706::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:03,708::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:03,794::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:57:03,902::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:03,936::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131641 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:05,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:06,000::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0208714 s, 49.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:08,021::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:08,055::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133311 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:08,438::__init__::232::IOProcess::(_processLogs) (8031) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:57:08,439::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:57:08,458::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000414898 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:08,460::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:08,460::__init__::232::IOProcess::(_processLogs) (8032) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:08,461::__init__::232::IOProcess::(_processLogs) DEBUG|(8032) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:08,462::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:08,463::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:08,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:08,464::__init__::232::IOProcess::(_processLogs) (8033) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:08,464::__init__::232::IOProcess::(_processLogs) (8033) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:08,464::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:08,465::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:08,466::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:08,466::__init__::232::IOProcess::(_processLogs) (8034) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:08,466::__init__::232::IOProcess::(_processLogs) (8034) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:08,466::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:08,467::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:08,468::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:08,469::__init__::232::IOProcess::(_processLogs) (8035) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:08,469::__init__::232::IOProcess::(_processLogs) (8035) Queuing response >Thread-133::DEBUG::2014-08-11 15:57:08,757::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce3cd06c-6393-40b5-bd9c-3f924b4a1ad5`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:57:08,758::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 15:57:08,794::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414898', 'lastCheck': '0.3', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:57:08,794::task::1191::Storage.TaskManager.Task::(prepare) Task=`ce3cd06c-6393-40b5-bd9c-3f924b4a1ad5`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414898', 'lastCheck': '0.3', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:57:08,794::task::595::Storage.TaskManager.Task::(_updateState) Task=`ce3cd06c-6393-40b5-bd9c-3f924b4a1ad5`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:57:08,794::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:57:08,795::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:57:08,795::task::993::Storage.TaskManager.Task::(_decref) Task=`ce3cd06c-6393-40b5-bd9c-3f924b4a1ad5`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:08,800::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:08,802::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:08,803::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:57:08,814::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Detector thread::DEBUG::2014-08-11 15:57:08,950::protocoldetector::160::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.35.162.23:44598 >Detector thread::DEBUG::2014-08-11 15:57:08,951::protocoldetector::171::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.35.162.23:44598 >Detector thread::DEBUG::2014-08-11 15:57:08,952::protocoldetector::197::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.35.162.23:44598 >Detector thread::DEBUG::2014-08-11 15:57:08,952::BindingXMLRPC::1167::XmlDetector::(handleSocket) xml over http detected from ('10.35.162.23', 44598) >Thread-134::DEBUG::2014-08-11 15:57:08,954::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::DEBUG::2014-08-11 15:57:08,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`472b7809-2626-44a8-8f68-427e821d0e45`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:57:08,955::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-134::INFO::2014-08-11 15:57:08,968::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:57:08,968::task::1191::Storage.TaskManager.Task::(prepare) Task=`472b7809-2626-44a8-8f68-427e821d0e45`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:57:08,968::task::595::Storage.TaskManager.Task::(_updateState) Task=`472b7809-2626-44a8-8f68-427e821d0e45`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:57:08,969::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:57:08,969::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:57:08,969::task::993::Storage.TaskManager.Task::(_decref) Task=`472b7809-2626-44a8-8f68-427e821d0e45`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:57:08,979::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:57:08,980::task::595::Storage.TaskManager.Task::(_updateState) Task=`82482f81-e3d2-45b9-a5e1-ebe5ca543bad`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:57:08,980::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:57:08,981::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ff8142f4-a2e2-44da-8a36-431672fef02f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:57:08,981::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:57:08,981::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:57:08,982::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ff8142f4-a2e2-44da-8a36-431672fef02f`::Granted request >Thread-34::DEBUG::2014-08-11 15:57:08,982::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`82482f81-e3d2-45b9-a5e1-ebe5ca543bad`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:57:08,982::task::993::Storage.TaskManager.Task::(_decref) Task=`82482f81-e3d2-45b9-a5e1-ebe5ca543bad`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:57:09,002::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:09,002::task::1191::Storage.TaskManager.Task::(prepare) Task=`82482f81-e3d2-45b9-a5e1-ebe5ca543bad`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:09,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`82482f81-e3d2-45b9-a5e1-ebe5ca543bad`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:57:09,003::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:57:09,003::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:57:09,003::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:57:09,004::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:57:09,004::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:57:09,004::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:57:09,004::task::993::Storage.TaskManager.Task::(_decref) Task=`82482f81-e3d2-45b9-a5e1-ebe5ca543bad`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:57:10,078::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:10,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131412 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:12,133::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:12,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0141713 s, 72.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:13,805::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:13,894::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:13,895::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:57:14,193::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:14,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132944 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:16,250::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:16,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132753 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:18,306::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:18,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132889 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:18,469::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:18,470::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:18,472::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 15:57:18,472::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:57:18,473::__init__::232::IOProcess::(_processLogs) (8036) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:18,492::__init__::232::IOProcess::(_processLogs) (8036) Queuing response >Thread-16::DEBUG::2014-08-11 15:57:18,493::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000657786 s, 692 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:18,494::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:18,494::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:18,495::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:18,495::__init__::232::IOProcess::(_processLogs) (8037) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:18,495::__init__::232::IOProcess::(_processLogs) (8037) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:18,496::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:18,497::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:18,498::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:18,498::__init__::232::IOProcess::(_processLogs) (8038) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:18,498::__init__::232::IOProcess::(_processLogs) (8038) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:18,498::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:18,499::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:18,500::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:18,500::__init__::232::IOProcess::(_processLogs) (8039) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:18,500::__init__::232::IOProcess::(_processLogs) (8039) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:18,500::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:18,501::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:18,502::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:18,502::__init__::232::IOProcess::(_processLogs) (8040) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:18,503::__init__::232::IOProcess::(_processLogs) (8040) Queuing response >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:18,902::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:18,904::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:18,905::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:57:19,017::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-134::DEBUG::2014-08-11 15:57:19,018::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f6c5a17-6c71-44fe-a2de-02b03e84c1d8`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:57:19,019::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-134::INFO::2014-08-11 15:57:19,031::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:57:19,031::task::1191::Storage.TaskManager.Task::(prepare) Task=`6f6c5a17-6c71-44fe-a2de-02b03e84c1d8`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:57:19,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f6c5a17-6c71-44fe-a2de-02b03e84c1d8`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:57:19,032::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:57:19,032::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:57:19,032::task::993::Storage.TaskManager.Task::(_decref) Task=`6f6c5a17-6c71-44fe-a2de-02b03e84c1d8`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:57:19,043::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] flowID [5480b349] >Thread-34::DEBUG::2014-08-11 15:57:19,044::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c8310f9-11e0-4897-88d4-0c0c9e52bff2`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:57:19,044::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:57:19,045::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a31ffd0f-8989-4bfd-9a8f-8d72940e2bfb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:57:19,045::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:57:19,045::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:57:19,046::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`a31ffd0f-8989-4bfd-9a8f-8d72940e2bfb`::Granted request >Thread-34::DEBUG::2014-08-11 15:57:19,046::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6c8310f9-11e0-4897-88d4-0c0c9e52bff2`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:57:19,046::task::993::Storage.TaskManager.Task::(_decref) Task=`6c8310f9-11e0-4897-88d4-0c0c9e52bff2`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:57:19,066::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:19,066::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c8310f9-11e0-4897-88d4-0c0c9e52bff2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943892480', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:19,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c8310f9-11e0-4897-88d4-0c0c9e52bff2`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:57:19,067::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:57:19,067::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:57:19,067::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:57:19,068::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:57:19,068::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:57:19,068::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:57:19,068::task::993::Storage.TaskManager.Task::(_decref) Task=`6c8310f9-11e0-4897-88d4-0c0c9e52bff2`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:57:20,363::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:20,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133284 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:22,418::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:22,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132379 s, 77.4 MB/s\n'; <rc> = 0 >Thread-134::DEBUG::2014-08-11 15:57:23,908::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ad31d90-8c81-4cf4-b863-a28bc1f90b63`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:57:23,908::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:23,911::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::INFO::2014-08-11 15:57:23,997::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657786', 'lastCheck': '5.5', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:57:23,998::task::1191::Storage.TaskManager.Task::(prepare) Task=`2ad31d90-8c81-4cf4-b863-a28bc1f90b63`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000657786', 'lastCheck': '5.5', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:57:24,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`2ad31d90-8c81-4cf4-b863-a28bc1f90b63`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:57:24,001::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:24,001::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:57:24,002::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:57:24,003::task::993::Storage.TaskManager.Task::(_decref) Task=`2ad31d90-8c81-4cf4-b863-a28bc1f90b63`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:24,007::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:57:24,030::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:57:24,472::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:24,506::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132909 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:26,529::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:26,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132481 s, 77.3 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:57:27,190::task::595::Storage.TaskManager.Task::(_updateState) Task=`441593e9-8f55-4b9a-a30f-e5d92ef8f9fd`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:57:27,190::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:57:27,191::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:27,192::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:27,192::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:27,192::__init__::232::IOProcess::(_processLogs) (8041) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:57:27,193::__init__::232::IOProcess::(_processLogs) DEBUG|(8041) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:27,194::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:27,194::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:27,195::__init__::232::IOProcess::(_processLogs) DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:27,195::__init__::232::IOProcess::(_processLogs) (8042) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:57:27,195::__init__::232::IOProcess::(_processLogs) DEBUG|(8042) Queuing response >Thread-122::INFO::2014-08-11 15:57:27,196::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:57:27,197::task::1191::Storage.TaskManager.Task::(prepare) Task=`441593e9-8f55-4b9a-a30f-e5d92ef8f9fd`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 15:57:27,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`441593e9-8f55-4b9a-a30f-e5d92ef8f9fd`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:57:27,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:57:27,197::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:57:27,198::task::993::Storage.TaskManager.Task::(_decref) Task=`441593e9-8f55-4b9a-a30f-e5d92ef8f9fd`::ref 0 aborting False >Thread-14::DEBUG::2014-08-11 15:57:28,503::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:28,503::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:28,504::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:28,504::__init__::232::IOProcess::(_processLogs) (8043) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:28,504::__init__::232::IOProcess::(_processLogs) DEBUG|(8043) Queuing response >Thread-16::DEBUG::2014-08-11 15:57:28,505::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:57:28,524::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000483042 s, 942 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:28,526::__init__::232::IOProcess::(_processLogs) DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:28,526::__init__::232::IOProcess::(_processLogs) DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:28,526::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:28,527::__init__::232::IOProcess::(_processLogs) (8044) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:28,528::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:28,528::__init__::232::IOProcess::(_processLogs) (8045) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:28,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:28,530::__init__::232::IOProcess::(_processLogs) (8046) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:28,531::__init__::232::IOProcess::(_processLogs) (8047) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:57:28,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:28,619::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131647 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:29,019::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:29,020::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:57:29,081::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::DEBUG::2014-08-11 15:57:29,098::task::595::Storage.TaskManager.Task::(_updateState) Task=`dda349a9-97db-4cb4-8b29-d7afd40a4871`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:57:29,100::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:29,100::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::INFO::2014-08-11 15:57:29,114::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:57:29,114::task::1191::Storage.TaskManager.Task::(prepare) Task=`dda349a9-97db-4cb4-8b29-d7afd40a4871`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:57:29,114::task::595::Storage.TaskManager.Task::(_updateState) Task=`dda349a9-97db-4cb4-8b29-d7afd40a4871`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:57:29,114::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:57:29,115::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:57:29,115::task::993::Storage.TaskManager.Task::(_decref) Task=`dda349a9-97db-4cb4-8b29-d7afd40a4871`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:57:29,125::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:57:29,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cf1b71c-8afd-47e9-b6d9-9911d7894475`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:57:29,126::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:57:29,127::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f2b093b5-654f-414c-a978-a2d53441c1ff`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:57:29,127::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:57:29,128::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:57:29,128::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`f2b093b5-654f-414c-a978-a2d53441c1ff`::Granted request >Thread-34::DEBUG::2014-08-11 15:57:29,129::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2cf1b71c-8afd-47e9-b6d9-9911d7894475`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:57:29,129::task::993::Storage.TaskManager.Task::(_decref) Task=`2cf1b71c-8afd-47e9-b6d9-9911d7894475`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:57:29,149::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945072128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:29,149::task::1191::Storage.TaskManager.Task::(prepare) Task=`2cf1b71c-8afd-47e9-b6d9-9911d7894475`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945072128', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:29,149::task::595::Storage.TaskManager.Task::(_updateState) Task=`2cf1b71c-8afd-47e9-b6d9-9911d7894475`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:57:29,150::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:57:29,150::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:57:29,150::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:57:29,151::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:57:29,151::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:57:29,151::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:57:29,151::task::993::Storage.TaskManager.Task::(_decref) Task=`2cf1b71c-8afd-47e9-b6d9-9911d7894475`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:57:30,641::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:30,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133569 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:32,698::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:32,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0280353 s, 36.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:34,108::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:34,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:34,196::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:57:34,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:34,990::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132824 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:37,014::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:37,048::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131817 s, 77.7 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:38,533::__init__::232::IOProcess::(_processLogs) (8048) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:57:38,534::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:57:38,554::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000419732 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:38,596::__init__::232::IOProcess::(_processLogs) (8049) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:38,598::__init__::232::IOProcess::(_processLogs) (8050) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:38,599::__init__::232::IOProcess::(_processLogs) (8051) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:38,600::__init__::232::IOProcess::(_processLogs) (8052) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 15:57:39,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:39,104::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133056 s, 77.0 MB/s\n'; <rc> = 0 >Thread-134::DEBUG::2014-08-11 15:57:39,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`51165459-8b01-47bd-bea5-1d70f25b187a`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:57:39,127::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::DEBUG::2014-08-11 15:57:39,164::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::INFO::2014-08-11 15:57:39,197::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419732', 'lastCheck': '0.6', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:57:39,198::task::595::Storage.TaskManager.Task::(_updateState) Task=`af565390-07f6-435f-b7b2-4d20dd230040`::moving from state init -> state preparing >Thread-134::DEBUG::2014-08-11 15:57:39,198::task::1191::Storage.TaskManager.Task::(prepare) Task=`51165459-8b01-47bd-bea5-1d70f25b187a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000419732', 'lastCheck': '0.6', 'valid': True}} >Thread-133::INFO::2014-08-11 15:57:39,199::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-134::DEBUG::2014-08-11 15:57:39,199::task::595::Storage.TaskManager.Task::(_updateState) Task=`51165459-8b01-47bd-bea5-1d70f25b187a`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:57:39,201::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:57:39,201::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:57:39,201::task::993::Storage.TaskManager.Task::(_decref) Task=`51165459-8b01-47bd-bea5-1d70f25b187a`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:39,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:57:39,215::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:57:39,217::task::1191::Storage.TaskManager.Task::(prepare) Task=`af565390-07f6-435f-b7b2-4d20dd230040`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:39,218::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:57:39,219::task::595::Storage.TaskManager.Task::(_updateState) Task=`af565390-07f6-435f-b7b2-4d20dd230040`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:57:39,220::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:57:39,221::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:57:39,224::task::993::Storage.TaskManager.Task::(_decref) Task=`af565390-07f6-435f-b7b2-4d20dd230040`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:39,225::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:57:39,237::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:57:39,238::task::595::Storage.TaskManager.Task::(_updateState) Task=`50c84a7f-c74b-49d0-867d-529b7d900230`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:57:39,238::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:57:39,239::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`46c4ed4d-b3bb-44d3-b5f7-8dab92699b68`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:57:39,239::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:57:39,240::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:57:39,240::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`46c4ed4d-b3bb-44d3-b5f7-8dab92699b68`::Granted request >Thread-34::DEBUG::2014-08-11 15:57:39,240::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`50c84a7f-c74b-49d0-867d-529b7d900230`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:57:39,241::task::993::Storage.TaskManager.Task::(_decref) Task=`50c84a7f-c74b-49d0-867d-529b7d900230`::ref 1 aborting False >Thread-133::DEBUG::2014-08-11 15:57:39,248::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:57:39,267::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:39,268::task::1191::Storage.TaskManager.Task::(prepare) Task=`50c84a7f-c74b-49d0-867d-529b7d900230`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:39,268::task::595::Storage.TaskManager.Task::(_updateState) Task=`50c84a7f-c74b-49d0-867d-529b7d900230`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:57:39,268::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:57:39,268::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:57:39,269::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:57:39,269::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:57:39,269::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:57:39,269::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:57:39,270::task::993::Storage.TaskManager.Task::(_decref) Task=`50c84a7f-c74b-49d0-867d-529b7d900230`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:57:41,134::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:41,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132673 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:43,189::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:43,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013126 s, 78.0 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:44,234::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:44,235::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:44,236::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:57:45,244::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:45,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131853 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:47,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:47,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133035 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:48,602::__init__::232::IOProcess::(_processLogs) (8053) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:57:48,603::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:57:48,623::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000421981 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:48,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:48,625::__init__::232::IOProcess::(_processLogs) (8054) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:48,626::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:48,626::__init__::232::IOProcess::(_processLogs) (8055) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:48,627::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:48,628::__init__::232::IOProcess::(_processLogs) (8056) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:48,628::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:48,629::__init__::232::IOProcess::(_processLogs) (8057) Got request for method 'access' >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:49,243::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:57:49,290::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:57:49,298::task::595::Storage.TaskManager.Task::(_updateState) Task=`780216ce-2705-4a00-8ae5-cdb1ef7079b5`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:57:49,300::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:49,301::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:49,302::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:57:49,315::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:57:49,315::task::1191::Storage.TaskManager.Task::(prepare) Task=`780216ce-2705-4a00-8ae5-cdb1ef7079b5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:57:49,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`780216ce-2705-4a00-8ae5-cdb1ef7079b5`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:57:49,316::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:57:49,316::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:57:49,316::task::993::Storage.TaskManager.Task::(_decref) Task=`780216ce-2705-4a00-8ae5-cdb1ef7079b5`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:57:49,327::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:57:49,327::task::595::Storage.TaskManager.Task::(_updateState) Task=`17dcb4ca-b6bb-4803-b665-f379e4e73584`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:57:49,328::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:57:49,328::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`681b4ce4-6dea-449c-89f1-f919e84e196b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:57:49,329::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:57:49,329::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:57:49,329::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`681b4ce4-6dea-449c-89f1-f919e84e196b`::Granted request >Thread-34::DEBUG::2014-08-11 15:57:49,330::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`17dcb4ca-b6bb-4803-b665-f379e4e73584`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:57:49,330::task::993::Storage.TaskManager.Task::(_decref) Task=`17dcb4ca-b6bb-4803-b665-f379e4e73584`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:57:49,350::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:49,350::task::1191::Storage.TaskManager.Task::(prepare) Task=`17dcb4ca-b6bb-4803-b665-f379e4e73584`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:49,350::task::595::Storage.TaskManager.Task::(_updateState) Task=`17dcb4ca-b6bb-4803-b665-f379e4e73584`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:57:49,351::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:57:49,351::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:57:49,351::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:57:49,352::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:57:49,352::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:57:49,352::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:57:49,352::task::993::Storage.TaskManager.Task::(_decref) Task=`17dcb4ca-b6bb-4803-b665-f379e4e73584`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:57:49,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:49,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013319 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:51,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:51,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132204 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:53,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:53,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131836 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:54,305::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:54,306::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:57:54,372::task::595::Storage.TaskManager.Task::(_updateState) Task=`60a0dd35-0943-432b-9fb0-02ad8a483758`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:57:54,396::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 15:57:54,399::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421981', 'lastCheck': '5.8', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:54,400::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:57:54,401::task::1191::Storage.TaskManager.Task::(prepare) Task=`60a0dd35-0943-432b-9fb0-02ad8a483758`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000421981', 'lastCheck': '5.8', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:57:54,402::task::595::Storage.TaskManager.Task::(_updateState) Task=`60a0dd35-0943-432b-9fb0-02ad8a483758`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:57:54,403::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:57:54,404::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:57:54,404::task::993::Storage.TaskManager.Task::(_decref) Task=`60a0dd35-0943-432b-9fb0-02ad8a483758`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 15:57:54,423::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:57:55,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:55,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.020124 s, 50.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:57:57,601::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:57,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132803 s, 77.1 MB/s\n'; <rc> = 0 >Thread-16::DEBUG::2014-08-11 15:57:58,630::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:57:58,632::__init__::232::IOProcess::(_processLogs) (8058) Got request for method 'glob' >Thread-16::DEBUG::2014-08-11 15:57:58,640::fileSD::150::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a >Thread-14::DEBUG::2014-08-11 15:57:58,641::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:58,641::__init__::232::IOProcess::(_processLogs) (8059) Got request for method 'touch' >Thread-14::DEBUG::2014-08-11 15:57:58,642::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8059) Queuing response >Thread-16::DEBUG::2014-08-11 15:57:58,643::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-14::DEBUG::2014-08-11 15:57:58,643::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:58,644::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:58,644::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:58,644::__init__::232::IOProcess::(_processLogs) (8060) Got request for method 'readfile' >Thread-14::DEBUG::2014-08-11 15:57:58,645::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8060) Queuing response >Thread-16::DEBUG::2014-08-11 15:57:58,645::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=on-qa', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=ON_qa', 'POOL_DOMAINS=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=8470a2b2-c7d3-4ab9-a762-8f157b9ae674', 'REMOTE_PATH=10.35.160.108:/RHEV/onqa', 'ROLE=Master', 'SDUUID=b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f0e7b35ed8655703b385ea9bdabd94b92d880f8c'] >Thread-14::DEBUG::2014-08-11 15:57:58,647::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:58,647::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:58,647::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:58,648::__init__::232::IOProcess::(_processLogs) (8061) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:58,648::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8061) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:58,649::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:58,649::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:58,650::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:58,650::__init__::232::IOProcess::(_processLogs) (8062) Got request for method 'glob' >Thread-14::DEBUG::2014-08-11 15:57:58,650::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8062) Queuing response >Thread-16::DEBUG::2014-08-11 15:57:58,650::fileSD::608::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-16::INFO::2014-08-11 15:57:58,652::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_imageNS already registered >Thread-16::INFO::2014-08-11 15:57:58,652::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace b978003f-2a0d-4dbc-92b3-ec6ed86cab4a_volumeNS already registered >Thread-14::DEBUG::2014-08-11 15:57:58,653::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:58,653::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:58,653::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:58,654::__init__::232::IOProcess::(_processLogs) (8063) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:58,654::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8063) Queuing response >Thread-16::DEBUG::2014-08-11 15:57:58,655::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:57:58,675::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000420917 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:57:58,676::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:58,676::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:58,677::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:58,677::__init__::232::IOProcess::(_processLogs) (8064) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:57:58,677::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8064) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:58,678::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:57:58,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:57:58,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:57:58,679::__init__::232::IOProcess::(_processLogs) (8065) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:58,679::__init__::232::IOProcess::(_processLogs) (8065) Queuing response >Thread-14::DEBUG::2014-08-11 15:57:58,680::__init__::232::IOProcess::(_processLogs) (8066) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:57:58,681::__init__::232::IOProcess::(_processLogs) (8067) Got request for method 'access' >Thread-133::DEBUG::2014-08-11 15:57:59,365::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:57:59,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd682fc1-f2f6-4da6-abe8-6a5660b7d24b`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:57:59,397::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:59,408::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:59,410::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:57:59,411::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:57:59,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:57:59,412::task::1191::Storage.TaskManager.Task::(prepare) Task=`cd682fc1-f2f6-4da6-abe8-6a5660b7d24b`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:57:59,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`cd682fc1-f2f6-4da6-abe8-6a5660b7d24b`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:57:59,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:57:59,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:57:59,413::task::993::Storage.TaskManager.Task::(_decref) Task=`cd682fc1-f2f6-4da6-abe8-6a5660b7d24b`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:57:59,424::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:57:59,424::task::595::Storage.TaskManager.Task::(_updateState) Task=`221de572-67ee-4a1d-a18a-1937f56d6db7`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:57:59,425::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:57:59,425::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8781c6e1-6335-4d69-ab44-4031ab768e86`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:57:59,426::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:57:59,426::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:57:59,426::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`8781c6e1-6335-4d69-ab44-4031ab768e86`::Granted request >Thread-34::DEBUG::2014-08-11 15:57:59,427::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`221de572-67ee-4a1d-a18a-1937f56d6db7`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:57:59,427::task::993::Storage.TaskManager.Task::(_decref) Task=`221de572-67ee-4a1d-a18a-1937f56d6db7`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:57:59,447::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:59,447::task::1191::Storage.TaskManager.Task::(prepare) Task=`221de572-67ee-4a1d-a18a-1937f56d6db7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:57:59,448::task::595::Storage.TaskManager.Task::(_updateState) Task=`221de572-67ee-4a1d-a18a-1937f56d6db7`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:57:59,448::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:57:59,448::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:57:59,449::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:57:59,449::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:57:59,449::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:57:59,449::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:57:59,450::task::993::Storage.TaskManager.Task::(_decref) Task=`221de572-67ee-4a1d-a18a-1937f56d6db7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:57:59,656::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:57:59,692::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137149 s, 74.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:01,713::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:01,747::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132785 s, 77.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:03,770::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:03,804::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132484 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:04,418::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:04,419::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:04,497::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:58:05,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:05,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133347 s, 76.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:07,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:07,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131544 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:08,684::__init__::232::IOProcess::(_processLogs) (8068) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:58:08,685::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:58:08,708::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00144651 s, 315 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:08,710::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:08,710::__init__::232::IOProcess::(_processLogs) (8069) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:08,711::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:08,711::__init__::232::IOProcess::(_processLogs) (8070) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:08,712::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:08,713::__init__::232::IOProcess::(_processLogs) (8071) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:08,713::__init__::232::IOProcess::(_processLogs) (8071) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:08,714::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:08,714::__init__::232::IOProcess::(_processLogs) (8072) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:08,714::__init__::232::IOProcess::(_processLogs) (8072) Queuing response >Thread-133::DEBUG::2014-08-11 15:58:09,462::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:58:09,463::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a6337a1-abc4-4cd5-b551-f10f99048b8f`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:58:09,497::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:09,505::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:09,506::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:09,507::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:58:09,509::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:58:09,509::task::1191::Storage.TaskManager.Task::(prepare) Task=`7a6337a1-abc4-4cd5-b551-f10f99048b8f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:58:09,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`7a6337a1-abc4-4cd5-b551-f10f99048b8f`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:58:09,510::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:58:09,510::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:58:09,510::task::993::Storage.TaskManager.Task::(_decref) Task=`7a6337a1-abc4-4cd5-b551-f10f99048b8f`::ref 0 aborting False >Thread-134::DEBUG::2014-08-11 15:58:09,513::task::595::Storage.TaskManager.Task::(_updateState) Task=`16f90d3a-28f1-4b6a-b637-ecc52dade0cb`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:58:09,513::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-134::INFO::2014-08-11 15:58:09,513::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00144651', 'lastCheck': '0.8', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:58:09,514::task::1191::Storage.TaskManager.Task::(prepare) Task=`16f90d3a-28f1-4b6a-b637-ecc52dade0cb`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00144651', 'lastCheck': '0.8', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:58:09,514::task::595::Storage.TaskManager.Task::(_updateState) Task=`16f90d3a-28f1-4b6a-b637-ecc52dade0cb`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:58:09,514::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:58:09,514::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:58:09,514::task::993::Storage.TaskManager.Task::(_decref) Task=`16f90d3a-28f1-4b6a-b637-ecc52dade0cb`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:58:09,527::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:58:09,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`fa708ebb-9c15-4e66-bc98-5225aca22459`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:58:09,528::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:58:09,528::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`07945a7c-621b-435d-9ae7-17c073e25deb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:58:09,529::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:58:09,529::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:58:09,529::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`07945a7c-621b-435d-9ae7-17c073e25deb`::Granted request >Thread-34::DEBUG::2014-08-11 15:58:09,530::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fa708ebb-9c15-4e66-bc98-5225aca22459`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:58:09,530::task::993::Storage.TaskManager.Task::(_decref) Task=`fa708ebb-9c15-4e66-bc98-5225aca22459`::ref 1 aborting False >Thread-134::DEBUG::2014-08-11 15:58:09,539::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:58:09,563::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:09,564::task::1191::Storage.TaskManager.Task::(prepare) Task=`fa708ebb-9c15-4e66-bc98-5225aca22459`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:09,564::task::595::Storage.TaskManager.Task::(_updateState) Task=`fa708ebb-9c15-4e66-bc98-5225aca22459`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:58:09,564::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:58:09,564::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:58:09,565::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:58:09,565::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:58:09,565::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:58:09,565::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:58:09,566::task::993::Storage.TaskManager.Task::(_decref) Task=`fa708ebb-9c15-4e66-bc98-5225aca22459`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:58:09,941::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:09,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131214 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:11,997::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:12,030::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133606 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:14,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:14,087::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132463 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:14,510::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:14,511::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:14,597::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:58:16,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:16,143::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013226 s, 77.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:18,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:18,200::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132479 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:18,716::__init__::232::IOProcess::(_processLogs) (8073) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 15:58:18,717::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:58:18,736::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426046 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:18,738::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:18,738::__init__::232::IOProcess::(_processLogs) (8074) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:18,740::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:18,740::__init__::232::IOProcess::(_processLogs) (8075) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:18,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:18,741::__init__::232::IOProcess::(_processLogs) (8076) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:18,741::__init__::232::IOProcess::(_processLogs) (8076) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:18,742::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:18,743::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:18,743::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:18,744::__init__::232::IOProcess::(_processLogs) (8077) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:18,745::__init__::232::IOProcess::(_processLogs) (8077) Queuing response >Thread-134::DEBUG::2014-08-11 15:58:19,577::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::DEBUG::2014-08-11 15:58:19,578::task::595::Storage.TaskManager.Task::(_updateState) Task=`75d808da-1863-4640-9ab0-4ebcbd81b245`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:58:19,596::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:19,604::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:19,605::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:19,606::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::INFO::2014-08-11 15:58:19,609::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:58:19,609::task::1191::Storage.TaskManager.Task::(prepare) Task=`75d808da-1863-4640-9ab0-4ebcbd81b245`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:58:19,610::task::595::Storage.TaskManager.Task::(_updateState) Task=`75d808da-1863-4640-9ab0-4ebcbd81b245`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:58:19,610::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:58:19,610::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:58:19,610::task::993::Storage.TaskManager.Task::(_decref) Task=`75d808da-1863-4640-9ab0-4ebcbd81b245`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:58:19,620::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:58:19,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`337e8c67-2b5f-4de1-8b4a-6079c684803e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:58:19,621::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:58:19,625::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ca915d49-083e-4da1-a9e5-20dac1ae1fbc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:58:19,625::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:58:19,626::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:58:19,626::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`ca915d49-083e-4da1-a9e5-20dac1ae1fbc`::Granted request >Thread-34::DEBUG::2014-08-11 15:58:19,626::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`337e8c67-2b5f-4de1-8b4a-6079c684803e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:58:19,627::task::993::Storage.TaskManager.Task::(_decref) Task=`337e8c67-2b5f-4de1-8b4a-6079c684803e`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:58:19,647::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:19,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`337e8c67-2b5f-4de1-8b4a-6079c684803e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:19,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`337e8c67-2b5f-4de1-8b4a-6079c684803e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:58:19,648::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:58:19,648::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:58:19,648::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:58:19,648::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:58:19,649::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:58:19,649::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:58:19,649::task::993::Storage.TaskManager.Task::(_decref) Task=`337e8c67-2b5f-4de1-8b4a-6079c684803e`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:58:20,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:20,257::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133499 s, 76.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:22,280::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:22,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148969 s, 68.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:24,339::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:24,373::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132595 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:24,612::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:24,613::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:58:24,650::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cf62d59-259f-46be-91aa-98aa3600cf8a`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:58:24,698::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-134::INFO::2014-08-11 15:58:24,700::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426046', 'lastCheck': '6.0', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:58:24,701::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cf62d59-259f-46be-91aa-98aa3600cf8a`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000426046', 'lastCheck': '6.0', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:24,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:58:24,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cf62d59-259f-46be-91aa-98aa3600cf8a`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:58:24,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:58:24,705::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:58:24,706::task::993::Storage.TaskManager.Task::(_decref) Task=`3cf62d59-259f-46be-91aa-98aa3600cf8a`::ref 0 aborting False >Thread-134::DEBUG::2014-08-11 15:58:24,724::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:58:26,396::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:26,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013194 s, 77.6 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:58:27,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc72e4be-a5ae-4044-9474-fd0a323ac6c7`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:58:27,304::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:58:27,304::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:27,305::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:27,305::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:27,305::__init__::232::IOProcess::(_processLogs) (8078) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:58:27,306::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|(8078) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:27,307::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:27,308::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:58:27,308::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:58:27,309::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-122::DEBUG::2014-08-11 15:58:27,310::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc72e4be-a5ae-4044-9474-fd0a323ac6c7`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:58:27,311::__init__::232::IOProcess::(_processLogs) (8079) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:58:27,311::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc72e4be-a5ae-4044-9474-fd0a323ac6c7`::moving from state preparing -> state finished >Thread-14::DEBUG::2014-08-11 15:58:27,312::__init__::232::IOProcess::(_processLogs) (8079) Queuing response >Thread-122::DEBUG::2014-08-11 15:58:27,312::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:58:27,313::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:58:27,313::task::993::Storage.TaskManager.Task::(_decref) Task=`bc72e4be-a5ae-4044-9474-fd0a323ac6c7`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:58:28,453::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:28,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132694 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:28,745::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:28,746::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:28,746::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:28,746::__init__::232::IOProcess::(_processLogs) (8080) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:28,746::__init__::232::IOProcess::(_processLogs) (8080) Queuing response >Thread-16::DEBUG::2014-08-11 15:58:28,747::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:58:28,767::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000426722 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:28,768::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:28,769::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:28,769::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:28,769::__init__::232::IOProcess::(_processLogs) (8081) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:28,770::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|(8081) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:28,771::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:28,772::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:28,772::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:28,773::__init__::232::IOProcess::(_processLogs) (8082) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:28,773::__init__::232::IOProcess::(_processLogs) (8082) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:28,773::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:28,774::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:28,775::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:28,775::__init__::232::IOProcess::(_processLogs) (8083) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:28,775::__init__::232::IOProcess::(_processLogs) (8083) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:28,776::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:28,776::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:28,777::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:28,778::__init__::232::IOProcess::(_processLogs) (8084) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:28,778::__init__::232::IOProcess::(_processLogs) (8084) Queuing response >Thread-134::DEBUG::2014-08-11 15:58:29,661::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::DEBUG::2014-08-11 15:58:29,662::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3be7fc7-f33e-4e25-8d0a-8fadb2cdb658`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:58:29,696::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-134::INFO::2014-08-11 15:58:29,709::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:58:29,709::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3be7fc7-f33e-4e25-8d0a-8fadb2cdb658`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:58:29,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3be7fc7-f33e-4e25-8d0a-8fadb2cdb658`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:58:29,710::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:58:29,710::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:58:29,712::task::993::Storage.TaskManager.Task::(_decref) Task=`d3be7fc7-f33e-4e25-8d0a-8fadb2cdb658`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:29,716::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:29,722::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:29,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:58:29,731::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:58:29,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea1bc674-deea-410a-b2e1-a6069ac95dee`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:58:29,734::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:58:29,738::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c2492fca-819f-4a4c-9d0f-dece9faea021`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:58:29,738::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:58:29,738::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:58:29,739::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c2492fca-819f-4a4c-9d0f-dece9faea021`::Granted request >Thread-34::DEBUG::2014-08-11 15:58:29,739::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ea1bc674-deea-410a-b2e1-a6069ac95dee`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:58:29,740::task::993::Storage.TaskManager.Task::(_decref) Task=`ea1bc674-deea-410a-b2e1-a6069ac95dee`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:58:29,759::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:29,760::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea1bc674-deea-410a-b2e1-a6069ac95dee`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:29,760::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea1bc674-deea-410a-b2e1-a6069ac95dee`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:58:29,760::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:58:29,760::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:58:29,761::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:58:29,761::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:58:29,761::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:58:29,762::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:58:29,762::task::993::Storage.TaskManager.Task::(_decref) Task=`ea1bc674-deea-410a-b2e1-a6069ac95dee`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:58:30,508::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:30,558::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0273236 s, 37.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:32,596::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:32,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0315955 s, 32.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:34,736::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:34,737::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:34,797::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:58:34,904::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:34,938::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132071 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:36,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:36,995::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132446 s, 77.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:38,778::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:38,780::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:58:38,781::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:58:38,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:38,800::__init__::232::IOProcess::(_processLogs) (8085) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:38,801::__init__::232::IOProcess::(_processLogs) (8085) Queuing response >Thread-16::DEBUG::2014-08-11 15:58:38,802::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000531922 s, 855 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:38,803::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:38,804::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:38,804::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:38,804::__init__::232::IOProcess::(_processLogs) (8086) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:38,805::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|(8086) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:38,806::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:38,807::__init__::232::IOProcess::(_processLogs) (8076) Queuing responseDEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:38,807::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:38,807::__init__::232::IOProcess::(_processLogs) (8087) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:38,808::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8087) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:38,809::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:38,809::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:38,810::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:38,810::__init__::232::IOProcess::(_processLogs) (8088) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:38,810::__init__::232::IOProcess::(_processLogs) (8088) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:38,811::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:38,811::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:38,811::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:38,811::__init__::232::IOProcess::(_processLogs) (8089) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:38,812::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8089) Queuing response >Dummy-33::DEBUG::2014-08-11 15:58:39,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:39,051::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131944 s, 77.6 MB/s\n'; <rc> = 0 >Thread-134::DEBUG::2014-08-11 15:58:39,777::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::DEBUG::2014-08-11 15:58:39,780::task::595::Storage.TaskManager.Task::(_updateState) Task=`71f85bbd-2723-4576-9a1b-23f56b8aa302`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:58:39,796::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:39,804::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:39,806::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:39,807::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::INFO::2014-08-11 15:58:39,810::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:58:39,812::task::1191::Storage.TaskManager.Task::(prepare) Task=`71f85bbd-2723-4576-9a1b-23f56b8aa302`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:58:39,813::task::595::Storage.TaskManager.Task::(_updateState) Task=`71f85bbd-2723-4576-9a1b-23f56b8aa302`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:58:39,814::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:58:39,814::task::595::Storage.TaskManager.Task::(_updateState) Task=`74034e1a-0377-473b-80a7-f6640faa30db`::moving from state init -> state preparing >Thread-134::DEBUG::2014-08-11 15:58:39,815::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:58:39,816::task::993::Storage.TaskManager.Task::(_decref) Task=`71f85bbd-2723-4576-9a1b-23f56b8aa302`::ref 0 aborting False >Thread-133::INFO::2014-08-11 15:58:39,816::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 15:58:39,818::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000531922', 'lastCheck': '1.0', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:58:39,819::task::1191::Storage.TaskManager.Task::(prepare) Task=`74034e1a-0377-473b-80a7-f6640faa30db`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000531922', 'lastCheck': '1.0', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:58:39,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`74034e1a-0377-473b-80a7-f6640faa30db`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:58:39,819::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:58:39,819::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:58:39,819::task::993::Storage.TaskManager.Task::(_decref) Task=`74034e1a-0377-473b-80a7-f6640faa30db`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:58:39,829::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:58:39,830::task::595::Storage.TaskManager.Task::(_updateState) Task=`d11d9acd-90e5-4826-8238-7300dbe2be74`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:58:39,830::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:58:39,831::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eca85669-06d6-461d-8333-bb06eab20ad4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:58:39,831::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:58:39,831::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:58:39,832::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`eca85669-06d6-461d-8333-bb06eab20ad4`::Granted request >Thread-34::DEBUG::2014-08-11 15:58:39,832::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d11d9acd-90e5-4826-8238-7300dbe2be74`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:58:39,832::task::993::Storage.TaskManager.Task::(_decref) Task=`d11d9acd-90e5-4826-8238-7300dbe2be74`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:58:39,852::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:39,852::task::1191::Storage.TaskManager.Task::(prepare) Task=`d11d9acd-90e5-4826-8238-7300dbe2be74`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:39,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`d11d9acd-90e5-4826-8238-7300dbe2be74`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:58:39,853::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:58:39,853::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:58:39,853::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:58:39,854::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:58:39,854::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:58:39,854::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:58:39,854::task::993::Storage.TaskManager.Task::(_decref) Task=`d11d9acd-90e5-4826-8238-7300dbe2be74`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 15:58:39,877::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:58:41,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:41,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.01317 s, 77.8 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:43,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:43,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131837 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:44,814::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:44,815::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:44,897::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:58:45,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:45,220::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013472 s, 76.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:47,243::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:47,277::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135 s, 75.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:48,814::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:48,814::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:48,814::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:48,815::__init__::232::IOProcess::(_processLogs) (8090) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:48,815::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8090) Queuing response >Thread-16::DEBUG::2014-08-11 15:58:48,817::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:58:48,836::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00041561 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:48,837::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:48,838::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:48,838::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:48,838::__init__::232::IOProcess::(_processLogs) (8091) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:48,839::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8091) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:48,840::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:48,841::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:48,842::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:48,842::__init__::232::IOProcess::(_processLogs) (8092) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:48,842::__init__::232::IOProcess::(_processLogs) (8092) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:48,842::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:48,843::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:48,843::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:48,843::__init__::232::IOProcess::(_processLogs) (8093) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:48,844::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8093) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:48,845::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:48,845::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:48,846::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:48,847::__init__::232::IOProcess::(_processLogs) (8094) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:48,847::__init__::232::IOProcess::(_processLogs) (8094) Queuing response >Dummy-33::DEBUG::2014-08-11 15:58:49,300::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:49,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133021 s, 77.0 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 15:58:49,884::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:58:49,885::task::595::Storage.TaskManager.Task::(_updateState) Task=`58f25f0c-aae8-4993-b0dd-40caf2fd3240`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:58:49,896::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:49,904::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:49,906::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:49,907::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:58:49,910::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:58:49,910::task::1191::Storage.TaskManager.Task::(prepare) Task=`58f25f0c-aae8-4993-b0dd-40caf2fd3240`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:58:49,910::task::595::Storage.TaskManager.Task::(_updateState) Task=`58f25f0c-aae8-4993-b0dd-40caf2fd3240`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:58:49,911::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:58:49,911::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:58:49,911::task::993::Storage.TaskManager.Task::(_decref) Task=`58f25f0c-aae8-4993-b0dd-40caf2fd3240`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:58:49,921::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:58:49,922::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a0e34e4-028d-45c4-ad4e-edcb899b4971`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:58:49,922::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:58:49,923::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`94921220-e6db-47fe-85ea-7023c0d5ca51`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:58:49,923::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:58:49,924::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:58:49,924::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`94921220-e6db-47fe-85ea-7023c0d5ca51`::Granted request >Thread-34::DEBUG::2014-08-11 15:58:49,925::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2a0e34e4-028d-45c4-ad4e-edcb899b4971`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:58:49,925::task::993::Storage.TaskManager.Task::(_decref) Task=`2a0e34e4-028d-45c4-ad4e-edcb899b4971`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:58:49,944::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:49,945::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a0e34e4-028d-45c4-ad4e-edcb899b4971`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:58:49,945::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a0e34e4-028d-45c4-ad4e-edcb899b4971`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:58:49,945::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:58:49,946::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:58:49,946::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:58:49,946::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:58:49,946::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:58:49,947::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:58:49,947::task::993::Storage.TaskManager.Task::(_decref) Task=`2a0e34e4-028d-45c4-ad4e-edcb899b4971`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:58:51,357::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:51,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132563 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:53,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:53,447::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132462 s, 77.3 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:54,914::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:54,915::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:58:54,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`176876a2-2e4e-41cd-9f4b-362259df4ba8`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:58:54,998::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 15:58:54,999::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041561', 'lastCheck': '6.2', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:58:55,001::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:58:55,001::task::1191::Storage.TaskManager.Task::(prepare) Task=`176876a2-2e4e-41cd-9f4b-362259df4ba8`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00041561', 'lastCheck': '6.2', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:58:55,003::task::595::Storage.TaskManager.Task::(_updateState) Task=`176876a2-2e4e-41cd-9f4b-362259df4ba8`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:58:55,004::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:58:55,005::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:58:55,005::task::993::Storage.TaskManager.Task::(_decref) Task=`176876a2-2e4e-41cd-9f4b-362259df4ba8`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 15:58:55,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:58:55,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:55,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133144 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:58:57,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:57,560::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131257 s, 78.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:58,848::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:58,848::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:58,848::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:58,849::__init__::232::IOProcess::(_processLogs) (8095) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:58,849::__init__::232::IOProcess::(_processLogs) (8095) Queuing response >Thread-16::DEBUG::2014-08-11 15:58:58,850::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:58:58,870::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000417039 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:58:58,871::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:58,871::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:58,872::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:58,872::__init__::232::IOProcess::(_processLogs) (8096) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:58:58,872::__init__::232::IOProcess::(_processLogs) (8096) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:58,873::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:58,874::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:58,875::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:58,875::__init__::232::IOProcess::(_processLogs) (8097) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:58,875::__init__::232::IOProcess::(_processLogs) (8097) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:58,875::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:58,876::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:58,877::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:58,877::__init__::232::IOProcess::(_processLogs) (8098) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:58,877::__init__::232::IOProcess::(_processLogs) (8098) Queuing response >Thread-14::DEBUG::2014-08-11 15:58:58,878::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:58:58,878::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:58:58,879::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:58:58,880::__init__::232::IOProcess::(_processLogs) (8099) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:58:58,880::__init__::232::IOProcess::(_processLogs) (8099) Queuing response >Dummy-33::DEBUG::2014-08-11 15:58:59,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:58:59,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132222 s, 77.4 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 15:58:59,959::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:58:59,960::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cb0f426-ec57-49d3-8bda-94562d6f8a09`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:58:59,997::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-133::INFO::2014-08-11 15:59:00,010::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:00,012::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:59:00,012::task::1191::Storage.TaskManager.Task::(prepare) Task=`9cb0f426-ec57-49d3-8bda-94562d6f8a09`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:59:00,013::task::595::Storage.TaskManager.Task::(_updateState) Task=`9cb0f426-ec57-49d3-8bda-94562d6f8a09`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:59:00,015::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:59:00,016::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:59:00,018::task::993::Storage.TaskManager.Task::(_decref) Task=`9cb0f426-ec57-49d3-8bda-94562d6f8a09`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:00,018::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:00,021::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 15:59:00,032::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:59:00,033::task::595::Storage.TaskManager.Task::(_updateState) Task=`43ca3e2d-feab-49fa-a13c-454ddcf1fc19`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:59:00,033::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:59:00,034::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6e852fbe-11ce-4127-ab88-7b520d78f17a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:59:00,034::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:59:00,035::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:59:00,035::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`6e852fbe-11ce-4127-ab88-7b520d78f17a`::Granted request >Thread-34::DEBUG::2014-08-11 15:59:00,035::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`43ca3e2d-feab-49fa-a13c-454ddcf1fc19`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:59:00,036::task::993::Storage.TaskManager.Task::(_decref) Task=`43ca3e2d-feab-49fa-a13c-454ddcf1fc19`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:59:00,056::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:00,056::task::1191::Storage.TaskManager.Task::(prepare) Task=`43ca3e2d-feab-49fa-a13c-454ddcf1fc19`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:00,056::task::595::Storage.TaskManager.Task::(_updateState) Task=`43ca3e2d-feab-49fa-a13c-454ddcf1fc19`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:59:00,056::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:59:00,057::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:59:00,057::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:59:00,057::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:59:00,058::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:59:00,058::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:59:00,058::task::993::Storage.TaskManager.Task::(_decref) Task=`43ca3e2d-feab-49fa-a13c-454ddcf1fc19`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:59:01,638::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:01,672::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135346 s, 75.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:03,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:03,730::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133131 s, 76.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:05,024::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:05,025::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:05,097::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:59:05,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:05,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133607 s, 76.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:07,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:07,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013421 s, 76.3 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:08,880::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:08,881::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:08,881::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:08,881::__init__::232::IOProcess::(_processLogs) (8100) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:08,881::__init__::232::IOProcess::(_processLogs) (8100) Queuing response >Thread-16::DEBUG::2014-08-11 15:59:08,882::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:59:08,902::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000520425 s, 874 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:08,903::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:08,904::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:08,904::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:08,904::__init__::232::IOProcess::(_processLogs) (8101) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:08,905::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8101) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:08,906::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:08,907::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:08,908::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:08,908::__init__::232::IOProcess::(_processLogs) (8102) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:08,908::__init__::232::IOProcess::(_processLogs) (8102) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:08,909::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:08,909::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:08,910::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:08,910::__init__::232::IOProcess::(_processLogs) (8103) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:08,910::__init__::232::IOProcess::(_processLogs) (8103) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:08,911::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:08,911::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:08,912::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:08,913::__init__::232::IOProcess::(_processLogs) (8104) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:08,913::__init__::232::IOProcess::(_processLogs) (8104) Queuing response >Dummy-33::DEBUG::2014-08-11 15:59:09,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:09,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133081 s, 76.9 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 15:59:10,070::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:59:10,071::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ac18bc8-7853-4d61-ad15-93d5fc920a30`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:59:10,096::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:10,104::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:10,106::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:10,107::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:59:10,109::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:59:10,109::task::1191::Storage.TaskManager.Task::(prepare) Task=`5ac18bc8-7853-4d61-ad15-93d5fc920a30`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:59:10,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`5ac18bc8-7853-4d61-ad15-93d5fc920a30`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:59:10,110::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:59:10,110::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:59:10,110::task::993::Storage.TaskManager.Task::(_decref) Task=`5ac18bc8-7853-4d61-ad15-93d5fc920a30`::ref 0 aborting False >Thread-134::DEBUG::2014-08-11 15:59:10,119::task::595::Storage.TaskManager.Task::(_updateState) Task=`11b12465-ffaf-42ea-88b5-5f3ceb40786c`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:59:10,119::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-134::INFO::2014-08-11 15:59:10,120::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000520425', 'lastCheck': '1.2', 'valid': True}} >Thread-34::DEBUG::2014-08-11 15:59:10,122::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::DEBUG::2014-08-11 15:59:10,122::task::1191::Storage.TaskManager.Task::(prepare) Task=`11b12465-ffaf-42ea-88b5-5f3ceb40786c`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000520425', 'lastCheck': '1.2', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:59:10,123::task::595::Storage.TaskManager.Task::(_updateState) Task=`11b12465-ffaf-42ea-88b5-5f3ceb40786c`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:59:10,124::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-34::DEBUG::2014-08-11 15:59:10,124::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb2dbd80-9512-4e10-85ba-d53c5e9cf6fe`::moving from state init -> state preparing >Thread-134::DEBUG::2014-08-11 15:59:10,125::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:59:10,126::task::993::Storage.TaskManager.Task::(_decref) Task=`11b12465-ffaf-42ea-88b5-5f3ceb40786c`::ref 0 aborting False >Thread-34::INFO::2014-08-11 15:59:10,126::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:59:10,132::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2c9fa261-d0f8-42c4-a22d-ef41f68f6a71`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:59:10,133::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:59:10,136::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:59:10,138::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`2c9fa261-d0f8-42c4-a22d-ef41f68f6a71`::Granted request >Thread-34::DEBUG::2014-08-11 15:59:10,142::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cb2dbd80-9512-4e10-85ba-d53c5e9cf6fe`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:59:10,144::task::993::Storage.TaskManager.Task::(_decref) Task=`cb2dbd80-9512-4e10-85ba-d53c5e9cf6fe`::ref 1 aborting False >Thread-134::DEBUG::2014-08-11 15:59:10,161::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:59:10,166::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:10,167::task::1191::Storage.TaskManager.Task::(prepare) Task=`cb2dbd80-9512-4e10-85ba-d53c5e9cf6fe`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:10,167::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb2dbd80-9512-4e10-85ba-d53c5e9cf6fe`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:59:10,167::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:59:10,167::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:59:10,168::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:59:10,168::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:59:10,168::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:59:10,168::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:59:10,169::task::993::Storage.TaskManager.Task::(_decref) Task=`cb2dbd80-9512-4e10-85ba-d53c5e9cf6fe`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:59:11,923::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:11,961::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134044 s, 76.4 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:13,983::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:14,017::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132076 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:15,114::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:15,115::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:15,197::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:59:16,040::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:16,074::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132009 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:18,097::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:18,130::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133161 s, 76.9 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:18,914::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:18,914::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:18,914::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:18,915::__init__::232::IOProcess::(_processLogs) (8105) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:18,915::__init__::232::IOProcess::(_processLogs) (8105) Queuing response >Thread-16::DEBUG::2014-08-11 15:59:18,916::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:59:18,935::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000438987 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:18,937::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:18,937::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:18,937::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:18,937::__init__::232::IOProcess::(_processLogs) (8106) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:18,938::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8106) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:18,939::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:18,940::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:18,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:18,941::__init__::232::IOProcess::(_processLogs) (8107) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:18,941::__init__::232::IOProcess::(_processLogs) (8107) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:18,942::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:18,942::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:18,943::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:18,943::__init__::232::IOProcess::(_processLogs) (8108) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:18,943::__init__::232::IOProcess::(_processLogs) (8108) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:18,944::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:18,944::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:18,945::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:18,946::__init__::232::IOProcess::(_processLogs) (8109) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:18,946::__init__::232::IOProcess::(_processLogs) (8109) Queuing response >Dummy-33::DEBUG::2014-08-11 15:59:20,153::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-134::DEBUG::2014-08-11 15:59:20,182::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Dummy-33::DEBUG::2014-08-11 15:59:20,187::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.014103 s, 72.6 MB/s\n'; <rc> = 0 >Thread-134::DEBUG::2014-08-11 15:59:20,198::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6fad76a-c60f-48df-8871-1276a3ca356f`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:59:20,200::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:20,206::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:20,209::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:20,212::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::INFO::2014-08-11 15:59:20,219::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:59:20,220::task::1191::Storage.TaskManager.Task::(prepare) Task=`d6fad76a-c60f-48df-8871-1276a3ca356f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:59:20,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`d6fad76a-c60f-48df-8871-1276a3ca356f`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:59:20,222::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:59:20,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:59:20,224::task::993::Storage.TaskManager.Task::(_decref) Task=`d6fad76a-c60f-48df-8871-1276a3ca356f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:59:20,238::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:59:20,240::task::595::Storage.TaskManager.Task::(_updateState) Task=`83f6d1b5-bd77-417c-b34a-1e44c7ca0c39`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:59:20,241::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:59:20,244::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c676e104-d7a7-4d8d-a2d4-3b2d17825897`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:59:20,245::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:59:20,247::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:59:20,248::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`c676e104-d7a7-4d8d-a2d4-3b2d17825897`::Granted request >Thread-34::DEBUG::2014-08-11 15:59:20,250::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`83f6d1b5-bd77-417c-b34a-1e44c7ca0c39`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:59:20,251::task::993::Storage.TaskManager.Task::(_decref) Task=`83f6d1b5-bd77-417c-b34a-1e44c7ca0c39`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:59:20,288::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:20,288::task::1191::Storage.TaskManager.Task::(prepare) Task=`83f6d1b5-bd77-417c-b34a-1e44c7ca0c39`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:20,289::task::595::Storage.TaskManager.Task::(_updateState) Task=`83f6d1b5-bd77-417c-b34a-1e44c7ca0c39`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:59:20,289::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:59:20,289::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:59:20,289::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:59:20,290::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:59:20,290::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:59:20,290::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:59:20,290::task::993::Storage.TaskManager.Task::(_decref) Task=`83f6d1b5-bd77-417c-b34a-1e44c7ca0c39`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:59:22,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:22,308::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131209 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:24,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:24,365::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131873 s, 77.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:25,220::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:25,221::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:59:25,255::task::595::Storage.TaskManager.Task::(_updateState) Task=`cca30c63-c48c-4e48-976a-4fba3b48cdb1`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:59:25,299::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-134::INFO::2014-08-11 15:59:25,301::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000438987', 'lastCheck': '6.4', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:25,302::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:59:25,302::task::1191::Storage.TaskManager.Task::(prepare) Task=`cca30c63-c48c-4e48-976a-4fba3b48cdb1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000438987', 'lastCheck': '6.4', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:59:25,304::task::595::Storage.TaskManager.Task::(_updateState) Task=`cca30c63-c48c-4e48-976a-4fba3b48cdb1`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:59:25,305::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:59:25,306::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:59:25,306::task::993::Storage.TaskManager.Task::(_decref) Task=`cca30c63-c48c-4e48-976a-4fba3b48cdb1`::ref 0 aborting False >Thread-134::DEBUG::2014-08-11 15:59:25,335::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:59:26,394::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:26,452::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0260135 s, 39.4 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 15:59:27,415::task::595::Storage.TaskManager.Task::(_updateState) Task=`028c4029-6ebe-4b22-bf1a-910edfd4a574`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 15:59:27,416::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 15:59:27,417::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:27,417::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:27,417::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:27,417::__init__::232::IOProcess::(_processLogs) (8110) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 15:59:27,418::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8110) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:27,419::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:27,420::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-122::INFO::2014-08-11 15:59:27,420::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:59:27,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:27,422::__init__::232::IOProcess::(_processLogs) (8111) Got request for method 'stat' >Thread-122::DEBUG::2014-08-11 15:59:27,422::task::1191::Storage.TaskManager.Task::(prepare) Task=`028c4029-6ebe-4b22-bf1a-910edfd4a574`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-14::DEBUG::2014-08-11 15:59:27,423::__init__::232::IOProcess::(_processLogs) (8111) Queuing response >Thread-122::DEBUG::2014-08-11 15:59:27,423::task::595::Storage.TaskManager.Task::(_updateState) Task=`028c4029-6ebe-4b22-bf1a-910edfd4a574`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 15:59:27,424::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 15:59:27,424::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 15:59:27,425::task::993::Storage.TaskManager.Task::(_decref) Task=`028c4029-6ebe-4b22-bf1a-910edfd4a574`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:59:28,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:28,667::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133335 s, 76.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:28,947::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:28,948::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 15:59:28,949::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 15:59:28,949::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:28,968::__init__::232::IOProcess::(_processLogs) (8112) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:28,969::__init__::232::IOProcess::(_processLogs) (8112) Queuing response >Thread-16::DEBUG::2014-08-11 15:59:28,970::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000513502 s, 886 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:28,971::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:28,972::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:28,972::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:28,972::__init__::232::IOProcess::(_processLogs) (8113) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:28,973::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8113) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:28,974::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:28,975::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:28,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:28,976::__init__::232::IOProcess::(_processLogs) (8114) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:28,976::__init__::232::IOProcess::(_processLogs) (8114) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:28,976::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:28,977::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:28,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:28,978::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8115) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:28,978::__init__::232::IOProcess::(_processLogs) (8115) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:28,979::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:28,979::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:28,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:28,981::__init__::232::IOProcess::(_processLogs) (8116) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:28,981::__init__::232::IOProcess::(_processLogs) (8116) Queuing response >Thread-134::DEBUG::2014-08-11 15:59:30,302::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::DEBUG::2014-08-11 15:59:30,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec1b3253-74e6-44d0-a5db-bf8b33e6135e`::moving from state init -> state preparing >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:30,308::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::INFO::2014-08-11 15:59:30,398::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:30,401::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:30,402::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::INFO::2014-08-11 15:59:30,412::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:59:30,413::task::1191::Storage.TaskManager.Task::(prepare) Task=`ec1b3253-74e6-44d0-a5db-bf8b33e6135e`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-134::DEBUG::2014-08-11 15:59:30,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec1b3253-74e6-44d0-a5db-bf8b33e6135e`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-08-11 15:59:30,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:59:30,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:59:30,414::task::993::Storage.TaskManager.Task::(_decref) Task=`ec1b3253-74e6-44d0-a5db-bf8b33e6135e`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:59:30,425::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:59:30,426::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d885735-2edd-481d-b560-b8dc0750a888`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:59:30,426::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:59:30,427::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e0e1673d-8438-4c3c-9756-a41090242ded`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:59:30,427::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:59:30,427::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:59:30,428::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`e0e1673d-8438-4c3c-9756-a41090242ded`::Granted request >Thread-34::DEBUG::2014-08-11 15:59:30,428::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5d885735-2edd-481d-b560-b8dc0750a888`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:59:30,429::task::993::Storage.TaskManager.Task::(_decref) Task=`5d885735-2edd-481d-b560-b8dc0750a888`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:59:30,448::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:30,449::task::1191::Storage.TaskManager.Task::(prepare) Task=`5d885735-2edd-481d-b560-b8dc0750a888`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943826944', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:30,449::task::595::Storage.TaskManager.Task::(_updateState) Task=`5d885735-2edd-481d-b560-b8dc0750a888`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:59:30,449::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:59:30,450::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:59:30,450::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:59:30,450::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:59:30,451::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:59:30,451::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:59:30,451::task::993::Storage.TaskManager.Task::(_decref) Task=`5d885735-2edd-481d-b560-b8dc0750a888`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:59:30,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:30,726::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.015015 s, 68.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:32,749::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:32,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.016766 s, 61.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:34,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:34,996::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0181563 s, 56.4 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:35,410::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:35,411::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:35,412::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:59:37,019::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:37,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132659 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:38,982::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:38,982::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:38,983::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:38,983::__init__::232::IOProcess::(_processLogs) (8117) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:38,983::__init__::232::IOProcess::(_processLogs) (8117) Queuing response >Thread-16::DEBUG::2014-08-11 15:59:38,984::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:59:39,004::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000424407 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:39,005::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:39,005::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:39,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:39,006::__init__::232::IOProcess::(_processLogs) (8118) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:39,006::__init__::232::IOProcess::(_processLogs) (8118) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:39,007::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:39,007::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:39,008::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:39,008::__init__::232::IOProcess::(_processLogs) (8119) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:39,008::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8119) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:39,009::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:39,010::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:39,011::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:39,011::__init__::232::IOProcess::(_processLogs) (8120) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:39,011::__init__::232::IOProcess::(_processLogs) (8120) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:39,011::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:39,012::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:39,012::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:39,013::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8121) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:39,013::__init__::232::IOProcess::(_processLogs) (8121) Queuing response >Dummy-33::DEBUG::2014-08-11 15:59:39,076::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:39,109::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131618 s, 77.8 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:40,415::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:40,416::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:59:40,434::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba9aff87-6bd7-4f97-8739-41f542b73211`::moving from state init -> state preparing >Thread-133::DEBUG::2014-08-11 15:59:40,463::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-134::INFO::2014-08-11 15:59:40,499::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::DEBUG::2014-08-11 15:59:40,501::task::595::Storage.TaskManager.Task::(_updateState) Task=`12c7c80d-f06f-468b-8029-1c7032433885`::moving from state init -> state preparing >Thread-134::INFO::2014-08-11 15:59:40,501::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424407', 'lastCheck': '1.5', 'valid': True}} >Thread-133::INFO::2014-08-11 15:59:40,503::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-134::DEBUG::2014-08-11 15:59:40,504::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba9aff87-6bd7-4f97-8739-41f542b73211`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000424407', 'lastCheck': '1.5', 'valid': True}} >Thread-134::DEBUG::2014-08-11 15:59:40,505::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba9aff87-6bd7-4f97-8739-41f542b73211`::moving from state preparing -> state finished >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:40,506::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-134::DEBUG::2014-08-11 15:59:40,507::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-134::DEBUG::2014-08-11 15:59:40,509::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-08-11 15:59:40,509::task::993::Storage.TaskManager.Task::(_decref) Task=`ba9aff87-6bd7-4f97-8739-41f542b73211`::ref 0 aborting False >Thread-133::INFO::2014-08-11 15:59:40,524::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:59:40,524::task::1191::Storage.TaskManager.Task::(prepare) Task=`12c7c80d-f06f-468b-8029-1c7032433885`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:59:40,525::task::595::Storage.TaskManager.Task::(_updateState) Task=`12c7c80d-f06f-468b-8029-1c7032433885`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:59:40,525::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:59:40,525::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:59:40,525::task::993::Storage.TaskManager.Task::(_decref) Task=`12c7c80d-f06f-468b-8029-1c7032433885`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:59:40,537::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:59:40,537::task::595::Storage.TaskManager.Task::(_updateState) Task=`69e5abcc-82e7-4754-9536-f44553bbc6ec`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:59:40,538::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:59:40,538::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d6b70b9e-51ae-46fa-a5b7-fc8397f42e50`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:59:40,539::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:59:40,539::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:59:40,539::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`d6b70b9e-51ae-46fa-a5b7-fc8397f42e50`::Granted request >Thread-34::DEBUG::2014-08-11 15:59:40,540::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`69e5abcc-82e7-4754-9536-f44553bbc6ec`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:59:40,540::task::993::Storage.TaskManager.Task::(_decref) Task=`69e5abcc-82e7-4754-9536-f44553bbc6ec`::ref 1 aborting False >Thread-133::DEBUG::2014-08-11 15:59:40,547::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::INFO::2014-08-11 15:59:40,579::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:40,580::task::1191::Storage.TaskManager.Task::(prepare) Task=`69e5abcc-82e7-4754-9536-f44553bbc6ec`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:40,580::task::595::Storage.TaskManager.Task::(_updateState) Task=`69e5abcc-82e7-4754-9536-f44553bbc6ec`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:59:40,580::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:59:40,580::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:59:40,581::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:59:40,581::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:59:40,581::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:59:40,581::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:59:40,582::task::993::Storage.TaskManager.Task::(_decref) Task=`69e5abcc-82e7-4754-9536-f44553bbc6ec`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:59:41,132::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:41,166::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131357 s, 78.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:43,188::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:43,222::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132518 s, 77.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:45,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:45,278::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132621 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:45,515::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:45,516::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:45,598::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:59:47,301::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:47,335::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132817 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:49,015::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:49,015::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:49,015::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:49,016::__init__::232::IOProcess::(_processLogs) (8122) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:49,016::__init__::232::IOProcess::(_processLogs) (8122) Queuing response >Thread-16::DEBUG::2014-08-11 15:59:49,017::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:59:49,037::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000429342 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:49,038::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:49,038::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:49,039::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:49,039::__init__::232::IOProcess::(_processLogs) (8123) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:49,039::__init__::232::IOProcess::(_processLogs) (8123) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:49,040::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:49,041::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:49,041::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:49,042::__init__::232::IOProcess::(_processLogs) (8124) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:49,042::__init__::232::IOProcess::(_processLogs) (8124) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:49,042::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:49,043::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:49,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:49,043::__init__::232::IOProcess::(_processLogs) (8125) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:49,044::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8125) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:49,044::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:49,045::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:49,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:49,046::__init__::232::IOProcess::(_processLogs) (8126) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:49,047::__init__::232::IOProcess::(_processLogs) (8126) Queuing response >Dummy-33::DEBUG::2014-08-11 15:59:49,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:49,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132827 s, 77.1 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 15:59:50,595::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 15:59:50,596::task::595::Storage.TaskManager.Task::(_updateState) Task=`c13e96c7-bacc-4d06-8c13-d05a82ae8be6`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:59:50,596::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:50,605::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:50,607::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:50,608::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 15:59:50,610::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:59:50,611::task::1191::Storage.TaskManager.Task::(prepare) Task=`c13e96c7-bacc-4d06-8c13-d05a82ae8be6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 15:59:50,611::task::595::Storage.TaskManager.Task::(_updateState) Task=`c13e96c7-bacc-4d06-8c13-d05a82ae8be6`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:59:50,611::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:59:50,611::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:59:50,612::task::993::Storage.TaskManager.Task::(_decref) Task=`c13e96c7-bacc-4d06-8c13-d05a82ae8be6`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 15:59:50,622::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 15:59:50,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e9ace76-b333-4048-bb5b-abfb9320a5a5`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 15:59:50,623::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 15:59:50,623::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0a11b1f8-c5cf-4925-9bfb-8e17669cbae4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 15:59:50,624::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 15:59:50,624::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 15:59:50,624::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`0a11b1f8-c5cf-4925-9bfb-8e17669cbae4`::Granted request >Thread-34::DEBUG::2014-08-11 15:59:50,625::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5e9ace76-b333-4048-bb5b-abfb9320a5a5`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 15:59:50,625::task::993::Storage.TaskManager.Task::(_decref) Task=`5e9ace76-b333-4048-bb5b-abfb9320a5a5`::ref 1 aborting False >Thread-34::INFO::2014-08-11 15:59:50,645::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:50,645::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e9ace76-b333-4048-bb5b-abfb9320a5a5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852945006592', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 15:59:50,646::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e9ace76-b333-4048-bb5b-abfb9320a5a5`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 15:59:50,646::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 15:59:50,646::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 15:59:50,646::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 15:59:50,647::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 15:59:50,647::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 15:59:50,647::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 15:59:50,647::task::993::Storage.TaskManager.Task::(_decref) Task=`5e9ace76-b333-4048-bb5b-abfb9320a5a5`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 15:59:51,414::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:51,448::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133105 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:53,471::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:53,505::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013262 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 15:59:55,528::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:55,561::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131513 s, 77.9 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:55,615::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:55,616::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:59:55,656::task::595::Storage.TaskManager.Task::(_updateState) Task=`075cc638-fdd2-4288-aa43-6940524a7984`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 15:59:55,699::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 15:59:55,700::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429342', 'lastCheck': '6.7', 'valid': True}} >Thread-133::DEBUG::2014-08-11 15:59:55,702::task::1191::Storage.TaskManager.Task::(prepare) Task=`075cc638-fdd2-4288-aa43-6940524a7984`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000429342', 'lastCheck': '6.7', 'valid': True}} >GuestMonitor-on-qa::DEBUG::2014-08-11 15:59:55,702::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 15:59:55,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`075cc638-fdd2-4288-aa43-6940524a7984`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 15:59:55,704::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 15:59:55,706::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 15:59:55,707::task::993::Storage.TaskManager.Task::(_decref) Task=`075cc638-fdd2-4288-aa43-6940524a7984`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 15:59:55,728::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 15:59:57,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:57,617::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131628 s, 77.8 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:59,047::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:59,048::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:59,048::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:59,048::__init__::232::IOProcess::(_processLogs) (8127) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:59,048::__init__::232::IOProcess::(_processLogs) (8127) Queuing response >Thread-16::DEBUG::2014-08-11 15:59:59,049::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 15:59:59,069::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000499459 s, 911 kB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 15:59:59,070::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:59,071::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:59,071::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:59,071::__init__::232::IOProcess::(_processLogs) (8128) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 15:59:59,071::__init__::232::IOProcess::(_processLogs) (8128) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:59,072::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:59,073::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:59,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:59,074::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|(8129) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:59,074::__init__::232::IOProcess::(_processLogs) (8129) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:59,075::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:59,075::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:59,076::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:59,076::__init__::232::IOProcess::(_processLogs) (8130) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:59,076::__init__::232::IOProcess::(_processLogs) (8130) Queuing response >Thread-14::DEBUG::2014-08-11 15:59:59,077::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 15:59:59,077::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 15:59:59,078::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 15:59:59,079::__init__::232::IOProcess::(_processLogs) (8131) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 15:59:59,079::__init__::232::IOProcess::(_processLogs) (8131) Queuing response >Dummy-33::DEBUG::2014-08-11 15:59:59,639::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 15:59:59,673::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136289 s, 75.1 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 16:00:00,660::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 16:00:00,661::task::595::Storage.TaskManager.Task::(_updateState) Task=`4158a6e1-99b8-49c3-b785-92a89f08bbeb`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:00,698::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-133::INFO::2014-08-11 16:00:00,710::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:00,711::task::1191::Storage.TaskManager.Task::(prepare) Task=`4158a6e1-99b8-49c3-b785-92a89f08bbeb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:00,711::task::595::Storage.TaskManager.Task::(_updateState) Task=`4158a6e1-99b8-49c3-b785-92a89f08bbeb`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:00,711::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 16:00:00,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:00,714::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 16:00:00,715::task::993::Storage.TaskManager.Task::(_decref) Task=`4158a6e1-99b8-49c3-b785-92a89f08bbeb`::ref 0 aborting False >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:00,718::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:00,720::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-34::DEBUG::2014-08-11 16:00:00,728::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 16:00:00,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`28126f69-829e-42ef-bb83-b8216eb173f9`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 16:00:00,729::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 16:00:00,730::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`54cbf3b5-24ba-4066-a8b0-ceb3ff407f77`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 16:00:00,730::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 16:00:00,730::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 16:00:00,730::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`54cbf3b5-24ba-4066-a8b0-ceb3ff407f77`::Granted request >Thread-34::DEBUG::2014-08-11 16:00:00,731::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`28126f69-829e-42ef-bb83-b8216eb173f9`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 16:00:00,731::task::993::Storage.TaskManager.Task::(_decref) Task=`28126f69-829e-42ef-bb83-b8216eb173f9`::ref 1 aborting False >Thread-34::INFO::2014-08-11 16:00:00,751::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:00,752::task::1191::Storage.TaskManager.Task::(prepare) Task=`28126f69-829e-42ef-bb83-b8216eb173f9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:00,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`28126f69-829e-42ef-bb83-b8216eb173f9`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 16:00:00,752::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 16:00:00,753::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 16:00:00,753::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 16:00:00,753::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 16:00:00,753::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 16:00:00,754::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 16:00:00,754::task::993::Storage.TaskManager.Task::(_decref) Task=`28126f69-829e-42ef-bb83-b8216eb173f9`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 16:00:01,696::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:01,729::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132138 s, 77.5 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 16:00:03,753::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:03,787::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0137207 s, 74.6 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:05,726::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:05,727::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:05,798::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 16:00:05,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:05,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.013303 s, 77.0 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 16:00:07,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:07,899::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132767 s, 77.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:09,080::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 16:00:09,080::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 16:00:09,082::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-16::DEBUG::2014-08-11 16:00:09,082::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 16:00:09,083::__init__::232::IOProcess::(_processLogs) (8132) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 16:00:09,102::__init__::232::IOProcess::(_processLogs) (8132) Queuing response >Thread-16::DEBUG::2014-08-11 16:00:09,103::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000441721 s, 1.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:09,104::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 16:00:09,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 16:00:09,105::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:09,107::__init__::232::IOProcess::(_processLogs) (8133) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 16:00:09,107::__init__::232::IOProcess::(_processLogs) (8133) Queuing response >Thread-14::DEBUG::2014-08-11 16:00:09,107::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 16:00:09,108::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 16:00:09,109::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:09,109::__init__::232::IOProcess::(_processLogs) (8134) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 16:00:09,109::__init__::232::IOProcess::(_processLogs) (8134) Queuing response >Thread-14::DEBUG::2014-08-11 16:00:09,110::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 16:00:09,110::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 16:00:09,111::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:09,111::__init__::232::IOProcess::(_processLogs) (8135) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 16:00:09,111::__init__::232::IOProcess::(_processLogs) (8135) Queuing response >Thread-14::DEBUG::2014-08-11 16:00:09,112::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 16:00:09,112::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 16:00:09,113::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:09,114::__init__::232::IOProcess::(_processLogs) (8136) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 16:00:09,114::__init__::232::IOProcess::(_processLogs) (8136) Queuing response >Dummy-33::DEBUG::2014-08-11 16:00:09,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:09,956::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133475 s, 76.7 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 16:00:10,768::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 16:00:10,769::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8742f90-50fe-4937-b6cc-81f610c6485f`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:10,798::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:10,805::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:10,809::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:10,810::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 16:00:10,811::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:10,812::task::1191::Storage.TaskManager.Task::(prepare) Task=`c8742f90-50fe-4937-b6cc-81f610c6485f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:10,812::task::595::Storage.TaskManager.Task::(_updateState) Task=`c8742f90-50fe-4937-b6cc-81f610c6485f`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:10,813::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 16:00:10,813::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 16:00:10,813::task::993::Storage.TaskManager.Task::(_decref) Task=`c8742f90-50fe-4937-b6cc-81f610c6485f`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 16:00:10,825::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 16:00:10,825::task::595::Storage.TaskManager.Task::(_updateState) Task=`0034ec52-fc88-4de1-8073-c1983f80e27e`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 16:00:10,826::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 16:00:10,827::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`27b377e1-2a1a-4472-a38a-ecc64f43c6af`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 16:00:10,827::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 16:00:10,827::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 16:00:10,827::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`27b377e1-2a1a-4472-a38a-ecc64f43c6af`::Granted request >Thread-34::DEBUG::2014-08-11 16:00:10,828::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0034ec52-fc88-4de1-8073-c1983f80e27e`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 16:00:10,828::task::993::Storage.TaskManager.Task::(_decref) Task=`0034ec52-fc88-4de1-8073-c1983f80e27e`::ref 1 aborting False >Thread-133::DEBUG::2014-08-11 16:00:10,842::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dd392f0-9167-425a-95dd-be69506fb9ab`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:10,843::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 16:00:10,843::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000441721', 'lastCheck': '1.7', 'valid': True}} >Thread-133::DEBUG::2014-08-11 16:00:10,843::task::1191::Storage.TaskManager.Task::(prepare) Task=`0dd392f0-9167-425a-95dd-be69506fb9ab`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000441721', 'lastCheck': '1.7', 'valid': True}} >Thread-133::DEBUG::2014-08-11 16:00:10,844::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dd392f0-9167-425a-95dd-be69506fb9ab`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:10,844::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 16:00:10,844::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 16:00:10,844::task::993::Storage.TaskManager.Task::(_decref) Task=`0dd392f0-9167-425a-95dd-be69506fb9ab`::ref 0 aborting False >Thread-34::INFO::2014-08-11 16:00:10,849::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:10,850::task::1191::Storage.TaskManager.Task::(prepare) Task=`0034ec52-fc88-4de1-8073-c1983f80e27e`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:10,851::task::595::Storage.TaskManager.Task::(_updateState) Task=`0034ec52-fc88-4de1-8073-c1983f80e27e`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 16:00:10,853::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 16:00:10,853::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 16:00:10,854::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 16:00:10,854::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 16:00:10,854::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 16:00:10,855::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 16:00:10,855::task::993::Storage.TaskManager.Task::(_decref) Task=`0034ec52-fc88-4de1-8073-c1983f80e27e`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 16:00:10,871::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 16:00:11,977::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:12,028::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0289823 s, 35.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 16:00:14,064::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:14,098::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132076 s, 77.5 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:15,818::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:15,899::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:15,900::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 16:00:16,120::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:16,154::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134626 s, 76.1 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 16:00:18,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:18,211::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132922 s, 77.0 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:19,115::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 16:00:19,116::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-16::DEBUG::2014-08-11 16:00:19,117::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-14::DEBUG::2014-08-11 16:00:19,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:19,136::__init__::232::IOProcess::(_processLogs) (8137) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 16:00:19,137::__init__::232::IOProcess::(_processLogs) (8137) Queuing response >Thread-16::DEBUG::2014-08-11 16:00:19,138::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423526 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:19,139::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Receiving request... >Thread-14::DEBUG::2014-08-11 16:00:19,139::__init__::232::IOProcess::(_processLogs) Extracting request information...DEBUG|Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 16:00:19,139::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:19,140::__init__::232::IOProcess::(_processLogs) (8138) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 16:00:19,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:19,142::__init__::232::IOProcess::(_processLogs) (8139) Queuing response >Thread-14::DEBUG::2014-08-11 16:00:19,143::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:19,143::__init__::232::IOProcess::(_processLogs) (8140) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 16:00:19,144::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:19,144::__init__::232::IOProcess::(_processLogs) (8141) Queuing response >Dummy-33::DEBUG::2014-08-11 16:00:20,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:20,279::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0246485 s, 41.5 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 16:00:20,876::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 16:00:20,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2b78fc8-7979-4af2-9979-9f1b8320d750`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:20,898::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:20,909::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:20,911::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 16:00:20,911::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:20,913::task::1191::Storage.TaskManager.Task::(prepare) Task=`a2b78fc8-7979-4af2-9979-9f1b8320d750`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:20,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`a2b78fc8-7979-4af2-9979-9f1b8320d750`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:20,915::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:20,916::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 16:00:20,916::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 16:00:20,918::task::993::Storage.TaskManager.Task::(_decref) Task=`a2b78fc8-7979-4af2-9979-9f1b8320d750`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 16:00:20,930::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 16:00:20,931::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a3be049-4cbf-4516-bf08-53ea7ba884e1`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 16:00:20,931::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 16:00:20,932::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4a91b2f1-fc32-4826-a677-90befb8fd9ce`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 16:00:20,932::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 16:00:20,932::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 16:00:20,932::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`4a91b2f1-fc32-4826-a677-90befb8fd9ce`::Granted request >Thread-34::DEBUG::2014-08-11 16:00:20,933::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2a3be049-4cbf-4516-bf08-53ea7ba884e1`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 16:00:20,933::task::993::Storage.TaskManager.Task::(_decref) Task=`2a3be049-4cbf-4516-bf08-53ea7ba884e1`::ref 1 aborting False >Thread-34::INFO::2014-08-11 16:00:20,953::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:20,953::task::1191::Storage.TaskManager.Task::(prepare) Task=`2a3be049-4cbf-4516-bf08-53ea7ba884e1`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943761408', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:20,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`2a3be049-4cbf-4516-bf08-53ea7ba884e1`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 16:00:20,954::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 16:00:20,954::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 16:00:20,955::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 16:00:20,955::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 16:00:20,955::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 16:00:20,955::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 16:00:20,956::task::993::Storage.TaskManager.Task::(_decref) Task=`2a3be049-4cbf-4516-bf08-53ea7ba884e1`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 16:00:22,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:22,336::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131735 s, 77.7 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 16:00:24,358::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:24,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132575 s, 77.2 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:25,926::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:25,927::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:25,999::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::DEBUG::2014-08-11 16:00:26,004::task::595::Storage.TaskManager.Task::(_updateState) Task=`7867489c-c1ec-412e-a04f-a3320d877b30`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:26,004::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 16:00:26,004::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423526', 'lastCheck': '6.9', 'valid': True}} >Thread-133::DEBUG::2014-08-11 16:00:26,005::task::1191::Storage.TaskManager.Task::(prepare) Task=`7867489c-c1ec-412e-a04f-a3320d877b30`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000423526', 'lastCheck': '6.9', 'valid': True}} >Thread-133::DEBUG::2014-08-11 16:00:26,005::task::595::Storage.TaskManager.Task::(_updateState) Task=`7867489c-c1ec-412e-a04f-a3320d877b30`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:26,005::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 16:00:26,005::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 16:00:26,006::task::993::Storage.TaskManager.Task::(_decref) Task=`7867489c-c1ec-412e-a04f-a3320d877b30`::ref 0 aborting False >Thread-133::DEBUG::2014-08-11 16:00:26,029::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 16:00:26,416::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:26,449::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0134047 s, 76.4 MB/s\n'; <rc> = 0 >Thread-122::DEBUG::2014-08-11 16:00:27,527::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e41ddde-a2f9-4eb0-8568-d4f340f3710b`::moving from state init -> state preparing >Thread-122::INFO::2014-08-11 16:00:27,528::logUtils::44::dispatcher::(wrapper) Run and protect: getVolumeSize(sdUUID='b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', imgUUID='1ce118d2-5534-46a2-8322-8d4038f3cadc', volUUID='d69ba38d-0ec3-4f6e-91a8-12a91d61b861', options=None) >Thread-14::DEBUG::2014-08-11 16:00:27,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:27,529::__init__::232::IOProcess::(_processLogs) (8142) Got request for method 'stat' >Thread-14::DEBUG::2014-08-11 16:00:27,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:27,531::__init__::232::IOProcess::(_processLogs) (8143) Got request for method 'stat' >Thread-122::INFO::2014-08-11 16:00:27,532::logUtils::47::dispatcher::(wrapper) Run and protect: getVolumeSize, Return response: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 16:00:27,532::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e41ddde-a2f9-4eb0-8568-d4f340f3710b`::finished: {'truesize': '1169801216', 'apparentsize': '10737418240'} >Thread-122::DEBUG::2014-08-11 16:00:27,532::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e41ddde-a2f9-4eb0-8568-d4f340f3710b`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-08-11 16:00:27,533::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-08-11 16:00:27,533::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-08-11 16:00:27,533::task::993::Storage.TaskManager.Task::(_decref) Task=`4e41ddde-a2f9-4eb0-8568-d4f340f3710b`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 16:00:28,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:28,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132637 s, 77.2 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:29,146::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-14::DEBUG::2014-08-11 16:00:29,146::__init__::232::IOProcess::(_processLogs) (8144) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 16:00:29,147::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 16:00:29,167::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.000423485 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:29,168::__init__::232::IOProcess::(_processLogs) (8145) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 16:00:29,170::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:29,170::__init__::232::IOProcess::(_processLogs) (8146) Queuing response >Thread-14::DEBUG::2014-08-11 16:00:29,171::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:29,172::__init__::232::IOProcess::(_processLogs) (8147) Queuing response >Thread-14::DEBUG::2014-08-11 16:00:29,173::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:29,173::__init__::232::IOProcess::(_processLogs) (8148) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 16:00:30,525::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:30,559::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131333 s, 78.0 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 16:00:30,969::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 16:00:30,970::task::595::Storage.TaskManager.Task::(_updateState) Task=`59cfc3f8-8835-442f-9410-45f2b8c3169c`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:30,999::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:31,005::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:31,008::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:31,010::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 16:00:31,011::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:31,012::task::1191::Storage.TaskManager.Task::(prepare) Task=`59cfc3f8-8835-442f-9410-45f2b8c3169c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:31,012::task::595::Storage.TaskManager.Task::(_updateState) Task=`59cfc3f8-8835-442f-9410-45f2b8c3169c`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:31,012::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 16:00:31,012::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 16:00:31,013::task::993::Storage.TaskManager.Task::(_decref) Task=`59cfc3f8-8835-442f-9410-45f2b8c3169c`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 16:00:31,023::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 16:00:31,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbb95e90-545c-4b93-b25c-421b2e7ef308`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 16:00:31,024::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 16:00:31,025::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b63f2aeb-6ce8-41f2-91f5-20506058e3cf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 16:00:31,025::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 16:00:31,026::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 16:00:31,026::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`b63f2aeb-6ce8-41f2-91f5-20506058e3cf`::Granted request >Thread-34::DEBUG::2014-08-11 16:00:31,027::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`dbb95e90-545c-4b93-b25c-421b2e7ef308`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 16:00:31,027::task::993::Storage.TaskManager.Task::(_decref) Task=`dbb95e90-545c-4b93-b25c-421b2e7ef308`::ref 1 aborting False >Thread-34::INFO::2014-08-11 16:00:31,047::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943695872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:31,047::task::1191::Storage.TaskManager.Task::(prepare) Task=`dbb95e90-545c-4b93-b25c-421b2e7ef308`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 9L, 'domains': 'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a:Active', 'master_uuid': u'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 1}, 'dominfo': {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'status': 'Active', 'diskfree': '1852943695872', 'isoprefix': '', 'alerts': [], 'disktotal': '2163246104576', 'version': 3}}} >Thread-34::DEBUG::2014-08-11 16:00:31,048::task::595::Storage.TaskManager.Task::(_updateState) Task=`dbb95e90-545c-4b93-b25c-421b2e7ef308`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-08-11 16:00:31,048::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674': < ResourceRef 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', isValid: 'True' obj: 'None'>} >Thread-34::DEBUG::2014-08-11 16:00:31,048::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-08-11 16:00:31,049::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' >Thread-34::DEBUG::2014-08-11 16:00:31,049::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' (0 active users) >Thread-34::DEBUG::2014-08-11 16:00:31,049::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free, finding out if anyone is waiting for it. >Thread-34::DEBUG::2014-08-11 16:00:31,049::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674', Clearing records. >Thread-34::DEBUG::2014-08-11 16:00:31,050::task::993::Storage.TaskManager.Task::(_decref) Task=`dbb95e90-545c-4b93-b25c-421b2e7ef308`::ref 0 aborting False >Dummy-33::DEBUG::2014-08-11 16:00:32,582::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:32,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0289968 s, 35.3 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 16:00:34,793::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:34,827::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133421 s, 76.7 MB/s\n'; <rc> = 0 >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:36,013::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:36,015::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:36,099::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Dummy-33::DEBUG::2014-08-11 16:00:36,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:36,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131906 s, 77.6 MB/s\n'; <rc> = 0 >Dummy-33::DEBUG::2014-08-11 16:00:38,906::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:38,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0153856 s, 66.6 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:39,175::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-14::DEBUG::2014-08-11 16:00:39,175::__init__::232::IOProcess::(_processLogs) (8149) Got request for method 'statvfs' >Thread-16::DEBUG::2014-08-11 16:00:39,177::fileSD::259::Storage.Misc.excCmd::(getReadDelay) /usr/bin/dd if=/rhev/data-center/mnt/10.35.160.108:_RHEV_onqa/b978003f-2a0d-4dbc-92b3-ec6ed86cab4a/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-16::DEBUG::2014-08-11 16:00:39,197::fileSD::259::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n455 bytes (455 B) copied, 0.00043292 s, 1.1 MB/s\n'; <rc> = 0 >Thread-14::DEBUG::2014-08-11 16:00:39,198::__init__::232::IOProcess::(_processLogs) (8150) Got request for method 'statvfs' >Thread-14::DEBUG::2014-08-11 16:00:39,200::__init__::232::IOProcess::(_processLogs) (8151) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 16:00:39,201::__init__::232::IOProcess::(_processLogs) (8152) Got request for method 'access' >Thread-14::DEBUG::2014-08-11 16:00:39,202::__init__::232::IOProcess::(_processLogs) (8153) Got request for method 'access' >Dummy-33::DEBUG::2014-08-11 16:00:40,973::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/8470a2b2-c7d3-4ab9-a762-8f157b9ae674/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-33::DEBUG::2014-08-11 16:00:41,007::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132286 s, 77.4 MB/s\n'; <rc> = 0 >Thread-133::DEBUG::2014-08-11 16:00:41,062::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-133::DEBUG::2014-08-11 16:00:41,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb28892c-c57e-4e71-bc17-d06cf46f3276`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:41,099::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:41,107::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:41,108::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >GuestMonitor-on-qa::DEBUG::2014-08-11 16:00:41,109::vm::444::vm.Vm::(_getUserCpuTuneInfo) vmId=`e9a797ed-f982-4208-af2a-d6558ca6dfed`::Domain Metadata is not set >Thread-133::INFO::2014-08-11 16:00:41,111::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:41,112::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb28892c-c57e-4e71-bc17-d06cf46f3276`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 9L}} >Thread-133::DEBUG::2014-08-11 16:00:41,112::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb28892c-c57e-4e71-bc17-d06cf46f3276`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:41,112::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 16:00:41,112::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 16:00:41,113::task::993::Storage.TaskManager.Task::(_decref) Task=`fb28892c-c57e-4e71-bc17-d06cf46f3276`::ref 0 aborting False >Thread-34::DEBUG::2014-08-11 16:00:41,125::BindingXMLRPC::318::vds::(wrapper) client [10.35.162.23] >Thread-34::DEBUG::2014-08-11 16:00:41,126::task::595::Storage.TaskManager.Task::(_updateState) Task=`305dfa27-7afc-4c2f-a032-70d991929339`::moving from state init -> state preparing >Thread-34::INFO::2014-08-11 16:00:41,126::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='8470a2b2-c7d3-4ab9-a762-8f157b9ae674', options=None) >Thread-34::DEBUG::2014-08-11 16:00:41,127::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a8954f5-bb92-4d31-872d-4aae09f76249`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2610' at 'getStoragePoolInfo' >Thread-34::DEBUG::2014-08-11 16:00:41,127::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' for lock type 'shared' >Thread-34::DEBUG::2014-08-11 16:00:41,127::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674' is free. Now locking as 'shared' (1 active user) >Thread-34::DEBUG::2014-08-11 16:00:41,128::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674`ReqID=`9a8954f5-bb92-4d31-872d-4aae09f76249`::Granted request >Thread-34::DEBUG::2014-08-11 16:00:41,128::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`305dfa27-7afc-4c2f-a032-70d991929339`::_resourcesAcquired: Storage.8470a2b2-c7d3-4ab9-a762-8f157b9ae674 (shared) >Thread-34::DEBUG::2014-08-11 16:00:41,128::task::993::Storage.TaskManager.Task::(_decref) Task=`305dfa27-7afc-4c2f-a032-70d991929339`::ref 1 aborting False >Thread-133::DEBUG::2014-08-11 16:00:41,136::task::595::Storage.TaskManager.Task::(_updateState) Task=`589b1521-178e-4823-a231-ccf1a8b8e3f1`::moving from state init -> state preparing >Thread-133::INFO::2014-08-11 16:00:41,138::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-133::INFO::2014-08-11 16:00:41,140::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043292', 'lastCheck': '1.9', 'valid': True}} >Thread-133::DEBUG::2014-08-11 16:00:41,142::task::1191::Storage.TaskManager.Task::(prepare) Task=`589b1521-178e-4823-a231-ccf1a8b8e3f1`::finished: {'b978003f-2a0d-4dbc-92b3-ec6ed86cab4a': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00043292', 'lastCheck': '1.9', 'valid': True}} >Thread-133::DEBUG::2014-08-11 16:00:41,144::task::595::Storage.TaskManager.Task::(_updateState) Task=`589b1521-178e-4823-a231-ccf1a8b8e3f1`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-08-11 16:00:41,145::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-08-11 16:00:41,147::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-08-11 16:00:41,148::task::993::Storage.TaskManager.Task::(_decref) Task=`589b1521-178e-4823-a231-ccf1a8b8e3f1`::ref 0 aborting False >